[==========] Running 38 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 37 tests from ExternalMiniClusterTest
[ RUN ] ExternalMiniClusterTest.TestKerberosReacquire
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:47:52 dist-test-slave-g350 krb5kdc[21298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:47:52 dist-test-slave-g350 krb5kdc[21298](info): set up 2 sockets
Aug 06 07:47:52 dist-test-slave-g350 krb5kdc[21298](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 07:47:54.044497 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.066s user 0.005s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:47:54 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:47:54Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:47:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:47:54.294807 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45927
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45927 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:47:54.711313 21314 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:47:54.711755 21314 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:47:54.711944 21314 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:47:54.722019 21314 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:47:54.722133 21314 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:47:54.722256 21314 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:47:54.722342 21314 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:47:54.734648 21314 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45927
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:47:54.736932 21314 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:47:54.738883 21314 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:47:54.755026 21319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:47:54.754962 21320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:47:54.758060 21322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:47:54.758611 21314 server_base.cc:1061] running on GCE node
I20260806 07:47:54.760002 21314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:47:54.762373 21314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:47:54.763757 21314 hybrid_clock.cc:648] HybridClock initialized: now 1786002474763698 us; error 93 us; skew 500 ppm
I20260806 07:47:54.764338 21314 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:47:54 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002474, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:47:54.768096 21314 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:47:54.770594 21314 webserver.cc:533] Webserver started at http://127.20.202.254:41789/ using document root <none> and password file <none>
I20260806 07:47:54.771700 21314 fs_manager.cc:362] Metadata directory not provided
I20260806 07:47:54.771829 21314 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:47:54.772429 21314 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:47:54.776152 21314 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "6df50b98bbbc4686a431e9718d4697cb"
format_stamp: "Formatted at 2026-08-06 07:47:54 on dist-test-slave-g350"
I20260806 07:47:54.777510 21314 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "6df50b98bbbc4686a431e9718d4697cb"
format_stamp: "Formatted at 2026-08-06 07:47:54 on dist-test-slave-g350"
I20260806 07:47:54.785797 21314 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260806 07:47:54.792411 21329 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:47:54.794538 21314 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:47:54.794849 21314 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6df50b98bbbc4686a431e9718d4697cb"
format_stamp: "Formatted at 2026-08-06 07:47:54 on dist-test-slave-g350"
I20260806 07:47:54.795126 21314 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:47:54.820935 21314 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:47:54.826468 21314 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:47:54.827000 21314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:47:54.859851 21314 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45927
I20260806 07:47:54.859851 21380 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45927 every 8 connection(s)
I20260806 07:47:54.862617 21314 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:47:54.871389 21381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:47:54.871377 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21314
I20260806 07:47:54.871816 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:47:54.891446 21381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Bootstrap starting.
I20260806 07:47:54.897424 21381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Neither blocks nor log segments found. Creating new log.
I20260806 07:47:54.899070 21381 log.cc:826] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Log is configured to *not* fsync() on all Append() calls
I20260806 07:47:54.903263 21381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: No bootstrap required, opened a new log
Aug 06 07:47:54 dist-test-slave-g350 krb5kdc[21298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:47:54.910724 21381 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:47:54.911473 21381 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:47:54.911640 21381 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df50b98bbbc4686a431e9718d4697cb, State: Initialized, Role: FOLLOWER
I20260806 07:47:54.913012 21381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:47:54.913331 21381 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:47:54.913647 21381 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:47:54.913918 21381 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:47:54.916672 21381 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:47:54.917915 21381 leader_election.cc:304] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df50b98bbbc4686a431e9718d4697cb; no voters:
I20260806 07:47:54.918774 21381 leader_election.cc:290] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:47:54.918936 21386 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:47:54.923799 21386 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 LEADER]: Becoming Leader. State: Replica: 6df50b98bbbc4686a431e9718d4697cb, State: Running, Role: LEADER
I20260806 07:47:54.924583 21386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:47:54.927408 21381 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:47:54.933866 21388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6df50b98bbbc4686a431e9718d4697cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } } }
I20260806 07:47:54.934281 21388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: This master's current role is: LEADER
I20260806 07:47:54.933916 21387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df50b98bbbc4686a431e9718d4697cb. Latest consensus state: current_term: 1 leader_uuid: "6df50b98bbbc4686a431e9718d4697cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } } }
I20260806 07:47:54.934453 21387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: This master's current role is: LEADER
I20260806 07:47:54.937765 21397 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:47:54.950129 21397 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:47:54.966248 21397 catalog_manager.cc:1383] Generated new cluster ID: a0251b8d59114fc99ff9f62c055c72f3
I20260806 07:47:54.966435 21397 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:47:54.985352 21397 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:47:54.986825 21397 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:47:54.998562 21397 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Generated new TSK 0
I20260806 07:47:54.999521 21397 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:47:55.097826 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45927
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:47:55.469792 21409 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:47:55.470104 21409 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:47:55.470199 21409 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:47:55.479521 21409 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:47:55.479722 21409 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:47:55.491297 21409 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45927
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:47:55.492928 21409 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:47:55.494519 21409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:47:55.506160 21414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:47:55.506134 21415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:47:55.506807 21417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:47:55.507543 21409 server_base.cc:1061] running on GCE node
I20260806 07:47:55.508128 21409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:47:55.509125 21409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:47:55.510324 21409 hybrid_clock.cc:648] HybridClock initialized: now 1786002475510260 us; error 54 us; skew 500 ppm
I20260806 07:47:55.510665 21409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:47:55 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002475, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:47:55.514019 21409 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:47:55.515626 21409 webserver.cc:533] Webserver started at http://127.20.202.193:40683/ using document root <none> and password file <none>
I20260806 07:47:55.516345 21409 fs_manager.cc:362] Metadata directory not provided
I20260806 07:47:55.516477 21409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:47:55.516849 21409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:47:55.519268 21409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "85537ba8c8c548d3a38437f32416deeb"
format_stamp: "Formatted at 2026-08-06 07:47:55 on dist-test-slave-g350"
I20260806 07:47:55.520043 21409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "85537ba8c8c548d3a38437f32416deeb"
format_stamp: "Formatted at 2026-08-06 07:47:55 on dist-test-slave-g350"
I20260806 07:47:55.525313 21409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:47:55.529019 21424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:47:55.530573 21409 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:47:55.530773 21409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "85537ba8c8c548d3a38437f32416deeb"
format_stamp: "Formatted at 2026-08-06 07:47:55 on dist-test-slave-g350"
I20260806 07:47:55.531078 21409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:47:55.543583 21409 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:47:55.547876 21409 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:47:55.548172 21409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:47:55.549355 21409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:47:55.551446 21409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:47:55.551561 21409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:47:55.551749 21409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:47:55.551864 21409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:47:55.595625 21409 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37821
I20260806 07:47:55.595716 21536 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37821 every 8 connection(s)
I20260806 07:47:55.597316 21409 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:47:55.598847 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21409
I20260806 07:47:55.599094 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:47:55 dist-test-slave-g350 krb5kdc[21298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002475, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:47:55.621261 21537 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45927
I20260806 07:47:55.621793 21537 heartbeater.cc:461] Registering TS with master...
I20260806 07:47:55.622792 21537 heartbeater.cc:507] Master 127.20.202.254:45927 requested a full tablet report, sending...
I20260806 07:47:55.625855 21346 ts_manager.cc:194] Registered new tserver with Master: 85537ba8c8c548d3a38437f32416deeb (127.20.202.193:37821)
I20260806 07:47:55.627562 21346 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:47007
I20260806 07:47:55.630779 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:47:56.631676 21537 heartbeater.cc:499] Master 127.20.202.254:45927 was elected leader, sending a full tablet report...
Aug 06 07:48:09 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002489, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:48:09.770700 21325 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 07:48:10 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002490, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:48:10.516660 21420 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 07:48:11 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260806 07:48:11.664215 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21314
I20260806 07:48:11.666534 21376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:11.777802 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21314
I20260806 07:48:11.802106 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45927
--webserver_interface=127.20.202.254
--webserver_port=41789
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45927 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:48:12.197031 21544 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:12.197415 21544 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:12.197569 21544 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:12.206064 21544 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:12.206211 21544 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:12.206331 21544 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:12.206411 21544 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:12.218529 21544 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45927
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=41789
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:12.220259 21544 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:12.221865 21544 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:12.230360 21550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:12.231045 21552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:12.231202 21549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:12.232223 21544 server_base.cc:1061] running on GCE node
I20260806 07:48:12.232978 21544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:12.234544 21544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:12.235787 21544 hybrid_clock.cc:648] HybridClock initialized: now 1786002492235732 us; error 33 us; skew 500 ppm
I20260806 07:48:12.236240 21544 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:48:12 dist-test-slave-g350 krb5kdc[21298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002492, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:48:12.239586 21544 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:48:12.241338 21544 webserver.cc:533] Webserver started at http://127.20.202.254:41789/ using document root <none> and password file <none>
I20260806 07:48:12.242172 21544 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:12.242291 21544 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:12.247869 21544 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:48:12.250980 21559 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:12.252543 21544 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:12.252753 21544 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6df50b98bbbc4686a431e9718d4697cb"
format_stamp: "Formatted at 2026-08-06 07:47:54 on dist-test-slave-g350"
I20260806 07:48:12.253584 21544 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:12.267817 21544 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:12.272006 21544 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:12.272318 21544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:12.293465 21544 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45927
I20260806 07:48:12.293485 21610 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45927 every 8 connection(s)
I20260806 07:48:12.295383 21544 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestKerberosReacquire.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:12.303040 21611 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:12.303117 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21544
I20260806 07:48:12.306957 21611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Bootstrap starting.
Aug 06 07:48:12 dist-test-slave-g350 krb5kdc[21298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:48:12.327108 21611 log.cc:826] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:12.336722 21611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:12.337353 21611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Bootstrap complete.
I20260806 07:48:12.344373 21611 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:48:12.345031 21611 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df50b98bbbc4686a431e9718d4697cb, State: Initialized, Role: FOLLOWER
I20260806 07:48:12.345686 21611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:48:12.345888 21611 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:48:12.345973 21611 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:48:12.346105 21611 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:12.349550 21611 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:48:12.350057 21611 leader_election.cc:304] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df50b98bbbc4686a431e9718d4697cb; no voters:
I20260806 07:48:12.350610 21611 leader_election.cc:290] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 07:48:12.350826 21616 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:48:12.351507 21616 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [term 2 LEADER]: Becoming Leader. State: Replica: 6df50b98bbbc4686a431e9718d4697cb, State: Running, Role: LEADER
I20260806 07:48:12.352017 21616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } }
I20260806 07:48:12.352664 21611 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:12.354550 21617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6df50b98bbbc4686a431e9718d4697cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } } }
I20260806 07:48:12.354933 21617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:12.356283 21618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df50b98bbbc4686a431e9718d4697cb. Latest consensus state: current_term: 2 leader_uuid: "6df50b98bbbc4686a431e9718d4697cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df50b98bbbc4686a431e9718d4697cb" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45927 } } }
I20260806 07:48:12.356575 21618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:12.357138 21621 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:12.363852 21621 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:12.365464 21621 catalog_manager.cc:1295] Loaded cluster ID: a0251b8d59114fc99ff9f62c055c72f3
I20260806 07:48:12.365571 21621 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:12.369818 21621 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:12.371476 21621 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 6df50b98bbbc4686a431e9718d4697cb: Loaded TSK: 0
I20260806 07:48:12.373196 21621 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:12.677630 21576 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "85537ba8c8c548d3a38437f32416deeb" instance_seqno: 1786002475585084) as {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:54257; Asking this server to re-register.
I20260806 07:48:12.678589 21537 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:12.678857 21537 heartbeater.cc:507] Master 127.20.202.254:45927 requested a full tablet report, sending...
I20260806 07:48:12.680626 21576 ts_manager.cc:194] Registered new tserver with Master: 85537ba8c8c548d3a38437f32416deeb (127.20.202.193:37821)
I20260806 07:48:12.689479 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:48:12.689774 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21409
Aug 06 07:48:12 dist-test-slave-g350 krb5kdc[21298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:48:12.709537 21532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:12.803337 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21409
I20260806 07:48:12.823603 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21544
I20260806 07:48:12.824913 21606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:12.931864 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21544
2026-08-06T07:48:12Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestKerberosReacquire (21017 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/0
I20260806 07:48:12.979645 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--webserver_interface=127.20.202.254
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
W20260806 07:48:13.317229 21639 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:13.317566 21639 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:13.317728 21639 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:13.326223 21639 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:13.326326 21639 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:13.326423 21639 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:13.326510 21639 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:13.336740 21639 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:13.338569 21639 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:13.340528 21639 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:13.348768 21645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:13.348798 21644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:13.349731 21647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:13.350419 21639 server_base.cc:1061] running on GCE node
I20260806 07:48:13.351217 21639 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:13.351368 21639 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:13.351434 21639 hybrid_clock.cc:648] HybridClock initialized: now 1786002493351417 us; error 0 us; skew 500 ppm
I20260806 07:48:13.351711 21639 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:13.354488 21639 webserver.cc:533] Webserver started at http://127.20.202.254:42909/ using document root <none> and password file <none>
I20260806 07:48:13.355201 21639 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:13.355291 21639 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:13.355604 21639 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:13.358055 21639 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "6f79a352d4aa4102baa5b53a4388bfec"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:13.358801 21639 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "6f79a352d4aa4102baa5b53a4388bfec"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:13.364112 21639 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:48:13.367645 21652 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:13.369145 21639 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:13.369328 21639 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6f79a352d4aa4102baa5b53a4388bfec"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:13.369582 21639 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:13.396272 21639 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:13.397408 21639 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:13.397969 21639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:13.419943 21639 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:33129
I20260806 07:48:13.419965 21704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:33129 every 8 connection(s)
I20260806 07:48:13.421571 21639 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:13.426141 21705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:13.429708 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21639
I20260806 07:48:13.429965 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:48:13.431521 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45385
--webserver_interface=127.20.202.253
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
I20260806 07:48:13.433802 21705 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:13.453723 21705 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:13.455030 21654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111)
W20260806 07:48:13.456914 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45385: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111)
I20260806 07:48:13.506410 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } attempt: 1
W20260806 07:48:13.508688 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45385: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111)
I20260806 07:48:13.576933 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } attempt: 2
W20260806 07:48:13.579129 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45385: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111)
I20260806 07:48:13.670434 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } attempt: 3
W20260806 07:48:13.672561 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45385: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111)
W20260806 07:48:13.776758 21707 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:13.777189 21707 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:13.777352 21707 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:13.787124 21707 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:13.787235 21707 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:13.787339 21707 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:13.787444 21707 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:13.798199 21707 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:13.800041 21707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:13.802126 21707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:13.811159 21715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:13.811200 21716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:13.812150 21718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:13.812462 21707 server_base.cc:1061] running on GCE node
I20260806 07:48:13.813251 21707 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:13.813428 21707 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:13.813505 21707 hybrid_clock.cc:648] HybridClock initialized: now 1786002493813485 us; error 0 us; skew 500 ppm
I20260806 07:48:13.813903 21707 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:13.815786 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } attempt: 4
I20260806 07:48:13.817039 21707 webserver.cc:533] Webserver started at http://127.20.202.253:42499/ using document root <none> and password file <none>
W20260806 07:48:13.817965 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45385: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111)
I20260806 07:48:13.818216 21707 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:13.818360 21707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:13.818775 21707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:13.821338 21707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "8bbf61a6f4e641abaf398d261b2bf14a"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:13.822293 21707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "8bbf61a6f4e641abaf398d261b2bf14a"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:13.828207 21707 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:48:13.831910 21723 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:13.833406 21707 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:13.833732 21707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "8bbf61a6f4e641abaf398d261b2bf14a"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:13.833947 21707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:13.851042 21707 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:13.851935 21707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:13.852183 21707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:13.873677 21707 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:45385
I20260806 07:48:13.873713 21775 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:45385 every 8 connection(s)
I20260806 07:48:13.875404 21707 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:48:13.879914 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:13.885339 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21707
I20260806 07:48:13.885515 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:48:13.885860 21776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:13.886586 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36587
--webserver_interface=127.20.202.252
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
I20260806 07:48:13.899374 21776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:13.908794 21776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:13.910073 21726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
W20260806 07:48:13.911724 21776 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
I20260806 07:48:13.961058 21776 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 1
W20260806 07:48:13.963483 21776 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
I20260806 07:48:14.031760 21776 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 2
W20260806 07:48:14.034333 21776 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
I20260806 07:48:14.117239 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } attempt: 5
I20260806 07:48:14.124440 21705 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:14.125576 21776 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 3
W20260806 07:48:14.126930 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
W20260806 07:48:14.127667 21776 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
I20260806 07:48:14.178162 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 1
W20260806 07:48:14.180353 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
I20260806 07:48:14.248607 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 2
W20260806 07:48:14.250758 21705 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
W20260806 07:48:14.250584 21779 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:14.251027 21779 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:14.251150 21779 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:14.260746 21779 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:14.260918 21779 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:14.261265 21779 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:14.261340 21779 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:14.270931 21776 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 4
W20260806 07:48:14.273005 21776 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111)
I20260806 07:48:14.273353 21779 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:14.275307 21779 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:14.277482 21779 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:14.287649 21787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:14.287614 21786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:14.288019 21790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:14.288617 21779 server_base.cc:1061] running on GCE node
I20260806 07:48:14.289407 21779 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:14.289629 21779 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:14.289731 21779 hybrid_clock.cc:648] HybridClock initialized: now 1786002494289702 us; error 0 us; skew 500 ppm
I20260806 07:48:14.290095 21779 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:14.293028 21779 webserver.cc:533] Webserver started at http://127.20.202.252:42855/ using document root <none> and password file <none>
I20260806 07:48:14.293887 21779 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:14.294031 21779 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:14.294391 21779 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:14.297099 21779 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf"
format_stamp: "Formatted at 2026-08-06 07:48:14 on dist-test-slave-g350"
I20260806 07:48:14.297979 21779 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf"
format_stamp: "Formatted at 2026-08-06 07:48:14 on dist-test-slave-g350"
I20260806 07:48:14.303663 21779 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260806 07:48:14.307425 21795 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:14.308844 21779 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:48:14.309046 21779 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf"
format_stamp: "Formatted at 2026-08-06 07:48:14 on dist-test-slave-g350"
I20260806 07:48:14.309279 21779 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:14.326660 21779 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:14.327610 21779 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:14.327898 21779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:14.355679 21779 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:36587
I20260806 07:48:14.355755 21846 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:36587 every 8 connection(s)
I20260806 07:48:14.357017 21705 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 3
I20260806 07:48:14.357553 21779 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:48:14.364126 21847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:14.367460 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21779
I20260806 07:48:14.367717 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:48:14.372656 21847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:14.383256 21705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap starting.
I20260806 07:48:14.386307 21847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:14.388110 21705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:14.389807 21705 log.cc:826] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:14.393512 21705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: No bootstrap required, opened a new log
I20260806 07:48:14.401752 21705 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.402091 21705 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:14.402228 21705 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f79a352d4aa4102baa5b53a4388bfec, State: Initialized, Role: FOLLOWER
I20260806 07:48:14.403033 21705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.405310 21847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:14.405570 21856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.405910 21856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.407155 21705 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:14.418843 21847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Bootstrap starting.
I20260806 07:48:14.423319 21847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:14.424695 21847 log.cc:826] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:14.422143 21639 master_runner.cc:187] Error getting master registration for 127.20.202.253:45385: OK, instance_id { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" instance_seqno: 1786002493868066 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:48:14.430004 21639 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:14.433826 21867 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:14.434201 21867 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:48:14.437433 21847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: No bootstrap required, opened a new log
I20260806 07:48:14.444218 21847 raft_consensus.cc:359] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.444518 21847 raft_consensus.cc:385] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:14.444710 21847 raft_consensus.cc:740] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae6bbd5d2b4088b7e7338ef4fae5bf, State: Initialized, Role: FOLLOWER
I20260806 07:48:14.445413 21847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.446745 21870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.448189 21870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.448349 21847 sys_catalog.cc:565] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:48:14.463541 21881 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:14.463706 21881 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:48:14.465368 21779 master_runner.cc:187] Error getting master registration for 127.20.202.253:45385: OK, instance_id { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" instance_seqno: 1786002493868066 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:48:14.473129 21779 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:14.572258 21776 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } attempt: 5
W20260806 07:48:14.580495 21801 tablet.cc:2406] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:48:14.586009 21776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Bootstrap starting.
I20260806 07:48:14.590703 21776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:14.592810 21776 log.cc:826] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:14.596222 21776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: No bootstrap required, opened a new log
I20260806 07:48:14.604946 21776 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.605289 21776 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:14.605410 21776 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbf61a6f4e641abaf398d261b2bf14a, State: Initialized, Role: FOLLOWER
I20260806 07:48:14.606177 21776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.607259 21884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.607658 21884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.608592 21776 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:14.637984 21707 master_runner.cc:264] No leader master found from master 8bbf61a6f4e641abaf398d261b2bf14a
I20260806 07:48:14.638142 21707 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:14.640908 21895 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:14.641072 21895 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:48:14.648850 21870 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:48:14.649158 21870 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.651540 21870 leader_election.cc:290] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f79a352d4aa4102baa5b53a4388bfec (127.20.202.254:33129), 8bbf61a6f4e641abaf398d261b2bf14a (127.20.202.253:45385)
W20260806 07:48:14.652040 21658 tablet.cc:2406] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:48:14.654330 21680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f79a352d4aa4102baa5b53a4388bfec" is_pre_election: true
I20260806 07:48:14.655213 21680 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae6bbd5d2b4088b7e7338ef4fae5bf in term 0.
I20260806 07:48:14.656102 21750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" is_pre_election: true
I20260806 07:48:14.656620 21750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ae6bbd5d2b4088b7e7338ef4fae5bf in term 0.
I20260806 07:48:14.657415 21797 leader_election.cc:304] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ae6bbd5d2b4088b7e7338ef4fae5bf, 8bbf61a6f4e641abaf398d261b2bf14a; no voters:
I20260806 07:48:14.658586 21870 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:48:14.658782 21870 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:48:14.658888 21870 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:14.662338 21870 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.665810 21680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f79a352d4aa4102baa5b53a4388bfec"
I20260806 07:48:14.666085 21680 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:14.669890 21680 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae6bbd5d2b4088b7e7338ef4fae5bf in term 1.
I20260806 07:48:14.670557 21796 leader_election.cc:304] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ae6bbd5d2b4088b7e7338ef4fae5bf, 6f79a352d4aa4102baa5b53a4388bfec; no voters:
I20260806 07:48:14.675328 21750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbf61a6f4e641abaf398d261b2bf14a"
I20260806 07:48:14.675601 21750 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:14.679883 21750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ae6bbd5d2b4088b7e7338ef4fae5bf in term 1.
I20260806 07:48:14.680691 21870 leader_election.cc:290] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [CANDIDATE]: Term 1 election: Requested vote from peers 6f79a352d4aa4102baa5b53a4388bfec (127.20.202.254:33129), 8bbf61a6f4e641abaf398d261b2bf14a (127.20.202.253:45385)
I20260806 07:48:14.681001 21870 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:48:14.681169 21870 raft_consensus.cc:697] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 LEADER]: Becoming Leader. State: Replica: 32ae6bbd5d2b4088b7e7338ef4fae5bf, State: Running, Role: LEADER
I20260806 07:48:14.681788 21870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:14.699628 21897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ae6bbd5d2b4088b7e7338ef4fae5bf. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.700145 21897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:14.714340 21899 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:14.717321 21899 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:14.744773 21750 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 1 FOLLOWER]: Refusing update from remote peer 32ae6bbd5d2b4088b7e7338ef4fae5bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:48:14.746344 21897 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:48:14.753515 21680 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Refusing update from remote peer 32ae6bbd5d2b4088b7e7338ef4fae5bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:48:14.754782 21897 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:48:14.788107 21856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ae6bbd5d2b4088b7e7338ef4fae5bf. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.788532 21856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.792182 21900 mvcc.cc:204] Tried to move back new op lower bound from 7315466218457636864 to 7315466218234126336. Current Snapshot: MvccSnapshot[applied={T|T < 7315466218457636864}]
I20260806 07:48:14.796963 21902 mvcc.cc:204] Tried to move back new op lower bound from 7315466218457636864 to 7315466218234126336. Current Snapshot: MvccSnapshot[applied={T|T < 7315466218457636864}]
I20260806 07:48:14.806294 21856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.806547 21856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.808399 21897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.808643 21897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:14.808751 21884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ae6bbd5d2b4088b7e7338ef4fae5bf. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.810540 21899 catalog_manager.cc:1383] Generated new cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:14.810657 21899 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:14.813709 21884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.815165 21870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.815521 21870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:14.839320 21899 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:48:14.841209 21899 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:14.861940 21899 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Generated new TSK 0
I20260806 07:48:14.863181 21899 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:14.866269 21884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:14.866503 21884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:14.900462 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--time_source=system_unsync with env {}
W20260806 07:48:15.359894 21909 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:15.360271 21909 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:15.360385 21909 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:15.374924 21909 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:15.375168 21909 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:15.389453 21909 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:15.391125 21909 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:15.392933 21909 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:15.403826 21914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:15.404136 21917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:15.404489 21915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:15.404882 21909 server_base.cc:1061] running on GCE node
I20260806 07:48:15.405504 21909 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:15.405730 21909 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:15.405841 21909 hybrid_clock.cc:648] HybridClock initialized: now 1786002495405802 us; error 0 us; skew 500 ppm
I20260806 07:48:15.406152 21909 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:15.408615 21909 webserver.cc:533] Webserver started at http://127.20.202.193:38941/ using document root <none> and password file <none>
I20260806 07:48:15.409559 21909 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:15.409813 21909 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:15.410259 21909 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:15.412892 21909 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:15.414126 21909 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:15.420442 21909 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:48:15.424741 21922 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:15.426507 21909 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:48:15.426678 21909 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:15.426894 21909 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:15.436852 21867 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:15.436950 21867 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: loading cluster ID for follower catalog manager: success
I20260806 07:48:15.441016 21867 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: acquiring CA information for follower catalog manager: success
I20260806 07:48:15.443513 21867 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:15.444846 21909 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:15.445739 21909 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:15.445971 21909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:15.446899 21909 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:15.448513 21909 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:15.448607 21909 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:15.448691 21909 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:15.448746 21909 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:15.486865 21909 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35507
I20260806 07:48:15.486887 22034 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35507 every 8 connection(s)
I20260806 07:48:15.488346 21909 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:15.496758 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21909
I20260806 07:48:15.496999 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:48:15.504071 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--time_source=system_unsync with env {}
I20260806 07:48:15.525660 22036 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:15.525906 22037 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:15.526161 22036 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:15.526167 22037 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:15.525660 22035 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:15.527088 22035 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:15.527515 22037 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:15.527624 22035 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:15.527689 22036 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:15.530328 21740 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:15.530346 21669 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:15.530896 21812 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:15.533313 21812 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:38223
I20260806 07:48:15.643389 21895 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:15.643493 21895 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: loading cluster ID for follower catalog manager: success
I20260806 07:48:15.646907 21895 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: acquiring CA information for follower catalog manager: success
I20260806 07:48:15.648801 21895 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:48:15.892149 22045 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:15.892520 22045 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:15.892666 22045 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:15.902060 22045 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:15.902277 22045 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:48:15.912791 22045 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:15.914706 22045 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:15.916566 22045 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:15.926397 22051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:15.926427 22050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:15.927138 22053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:15.927600 22045 server_base.cc:1061] running on GCE node
I20260806 07:48:15.928203 22045 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:15.928349 22045 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:15.928417 22045 hybrid_clock.cc:648] HybridClock initialized: now 1786002495928399 us; error 0 us; skew 500 ppm
I20260806 07:48:15.928674 22045 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:15.930883 22045 webserver.cc:533] Webserver started at http://127.20.202.194:38285/ using document root <none> and password file <none>
I20260806 07:48:15.931619 22045 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:15.931715 22045 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:15.932014 22045 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:15.934657 22045 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "6676a466cf5b4cd8ba98d24a7d2d660e"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:15.935401 22045 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "6676a466cf5b4cd8ba98d24a7d2d660e"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:15.941504 22045 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 07:48:15.946648 22058 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:15.948467 22045 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:48:15.948668 22045 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "6676a466cf5b4cd8ba98d24a7d2d660e"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:15.948947 22045 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:15.984367 22045 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:15.985366 22045 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:15.985710 22045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:15.986728 22045 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:15.989014 22045 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:15.989141 22045 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:15.989293 22045 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:15.989388 22045 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:16.030114 22045 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:34471
I20260806 07:48:16.030205 22170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:34471 every 8 connection(s)
I20260806 07:48:16.031680 22045 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:48:16.033666 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22045
I20260806 07:48:16.033897 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:48:16.055797 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--time_source=system_unsync with env {}
I20260806 07:48:16.073694 22172 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:16.074084 22172 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:16.074483 22173 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:16.074702 22173 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:16.075398 22173 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:16.075493 22172 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:16.075525 22171 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:16.075704 22171 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:16.076280 22171 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:16.077538 21812 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:34471)
I20260806 07:48:16.077879 21740 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:34471)
I20260806 07:48:16.077904 21669 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:34471)
I20260806 07:48:16.078681 21812 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:53841
W20260806 07:48:16.435127 22181 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:16.435492 22181 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:16.435645 22181 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:16.445029 22181 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:16.445254 22181 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:48:16.456732 22181 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:16.458570 22181 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:16.460434 22181 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:16.470916 22187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:16.471014 22186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:16.471628 22181 server_base.cc:1061] running on GCE node
W20260806 07:48:16.470891 22189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:16.472472 22181 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:16.472606 22181 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:16.472671 22181 hybrid_clock.cc:648] HybridClock initialized: now 1786002496472652 us; error 0 us; skew 500 ppm
I20260806 07:48:16.472930 22181 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:16.475138 22181 webserver.cc:533] Webserver started at http://127.20.202.195:36745/ using document root <none> and password file <none>
I20260806 07:48:16.476009 22181 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:16.476125 22181 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:16.476459 22181 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:16.479067 22181 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "f197f2ce7d9e4359bfbf6ee094135742"
format_stamp: "Formatted at 2026-08-06 07:48:16 on dist-test-slave-g350"
I20260806 07:48:16.479833 22181 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "f197f2ce7d9e4359bfbf6ee094135742"
format_stamp: "Formatted at 2026-08-06 07:48:16 on dist-test-slave-g350"
I20260806 07:48:16.485350 22181 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:48:16.488976 22194 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:16.491055 22181 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260806 07:48:16.491236 22181 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "f197f2ce7d9e4359bfbf6ee094135742"
format_stamp: "Formatted at 2026-08-06 07:48:16 on dist-test-slave-g350"
I20260806 07:48:16.491456 22181 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:16.511958 22181 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:16.512957 22181 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:16.513258 22181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:16.514544 22181 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:16.516517 22181 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:16.516666 22181 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:16.516814 22181 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:16.516916 22181 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:16.536087 22035 heartbeater.cc:499] Master 127.20.202.252:36587 was elected leader, sending a full tablet report...
I20260806 07:48:16.559653 22181 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:35519
I20260806 07:48:16.559712 22306 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:35519 every 8 connection(s)
I20260806 07:48:16.561203 22181 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:48:16.563463 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22181
I20260806 07:48:16.563685 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:48:16.583065 22307 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:16.583477 22307 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:16.584352 22308 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:16.584497 22308 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:16.584504 22307 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:16.584966 22308 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:16.585345 22309 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:16.585474 22309 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:16.586077 22309 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:16.586824 21812 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:35519)
I20260806 07:48:16.587380 21740 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:35519)
I20260806 07:48:16.587847 21812 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:41445
I20260806 07:48:16.589087 21669 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:35519)
I20260806 07:48:16.593858 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:16.601303 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:16.601672 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21639
I20260806 07:48:16.602842 21700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:16.711800 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21639
I20260806 07:48:16.735327 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--webserver_interface=127.20.202.254
--webserver_port=42909
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
I20260806 07:48:17.081032 22171 heartbeater.cc:499] Master 127.20.202.252:36587 was elected leader, sending a full tablet report...
W20260806 07:48:17.081161 22172 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:33129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:33129: connect: Connection refused (error 111)
W20260806 07:48:17.098374 22315 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:17.098696 22315 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:17.098773 22315 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:17.107697 22315 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:17.107798 22315 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:17.107857 22315 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:17.107897 22315 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:17.118319 22315 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=42909
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:17.119959 22315 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:17.121815 22315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:17.131028 22324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:17.131072 22322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:17.131114 22321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:17.132375 22315 server_base.cc:1061] running on GCE node
I20260806 07:48:17.133301 22315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:17.133495 22315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:17.133582 22315 hybrid_clock.cc:648] HybridClock initialized: now 1786002497133564 us; error 0 us; skew 500 ppm
I20260806 07:48:17.134017 22315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:17.136948 22315 webserver.cc:533] Webserver started at http://127.20.202.254:42909/ using document root <none> and password file <none>
I20260806 07:48:17.137778 22315 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:17.137903 22315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:17.143977 22315 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:48:17.147639 22330 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:17.147554 21796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:33129: connect: Connection refused (error 111)
W20260806 07:48:17.149309 21796 consensus_peers.cc:597] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf -> Peer 6f79a352d4aa4102baa5b53a4388bfec (127.20.202.254:33129): Couldn't send request to peer 6f79a352d4aa4102baa5b53a4388bfec. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:33129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:48:17.149435 22315 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 07:48:17.149695 22315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6f79a352d4aa4102baa5b53a4388bfec"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:17.150475 22315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:17.177378 22315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:17.178331 22315 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:17.178694 22315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:17.199155 22315 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:33129
I20260806 07:48:17.199196 22382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:33129 every 8 connection(s)
I20260806 07:48:17.200858 22315 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:17.207336 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22315
I20260806 07:48:17.207713 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21909
I20260806 07:48:17.208488 22383 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:17.212903 22383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap starting.
I20260806 07:48:17.220755 22030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:17.235980 22383 log.cc:826] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:17.247984 22383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:17.248673 22383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap complete.
I20260806 07:48:17.259238 22383 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:17.260219 22383 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f79a352d4aa4102baa5b53a4388bfec, State: Initialized, Role: FOLLOWER
I20260806 07:48:17.261104 22383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:17.262058 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:17.262354 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:17.263412 22383 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:17.274720 22400 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:17.274946 22400 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: loading cluster ID for follower catalog manager: success
I20260806 07:48:17.283370 22400 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: acquiring CA information for follower catalog manager: success
I20260806 07:48:17.286454 22400 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:17.334985 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21909
I20260806 07:48:17.355894 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35507
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--webserver_port=38941
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
I20260806 07:48:17.590226 22307 heartbeater.cc:499] Master 127.20.202.252:36587 was elected leader, sending a full tablet report...
I20260806 07:48:17.603940 22348 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "f197f2ce7d9e4359bfbf6ee094135742" instance_seqno: 1786002496550152) as {username='slave'} at 127.20.202.195:43393; Asking this server to re-register.
I20260806 07:48:17.604884 22308 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:17.606841 22348 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:35519)
W20260806 07:48:17.714082 22406 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:17.714536 22406 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:17.714708 22406 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:48:17.715199 21870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:17.715464 21870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: LEADER
W20260806 07:48:17.732949 22406 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:17.733234 22406 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:17.734859 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ae6bbd5d2b4088b7e7338ef4fae5bf. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:17.735139 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:17.745971 22406 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35507
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38941
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:17.747797 22406 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:17.750015 22406 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:17.760821 22417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:17.760851 22416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:17.761574 22419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:17.762521 22406 server_base.cc:1061] running on GCE node
I20260806 07:48:17.763135 22406 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:17.763320 22406 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:17.763419 22406 hybrid_clock.cc:648] HybridClock initialized: now 1786002497763382 us; error 0 us; skew 500 ppm
I20260806 07:48:17.763753 22406 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:17.766337 22406 webserver.cc:533] Webserver started at http://127.20.202.193:38941/ using document root <none> and password file <none>
I20260806 07:48:17.767290 22406 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:17.767449 22406 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:17.773059 22406 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 07:48:17.776022 22424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:17.777504 22406 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:48:17.777709 22406 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:17.778424 22406 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:17.795713 22406 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:17.796717 22406 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:17.796981 22406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:17.798004 22406 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:17.799742 22406 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:17.799816 22406 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:17.799906 22406 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:17.800001 22406 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:17.836097 22406 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35507
I20260806 07:48:17.836133 22536 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35507 every 8 connection(s)
I20260806 07:48:17.837877 22406 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:17.839404 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22406
I20260806 07:48:17.839810 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22406
I20260806 07:48:17.870651 22537 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:17.870750 22538 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:17.871063 22538 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:17.871068 22537 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:17.872363 22538 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:17.872591 22537 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:17.873227 22539 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:17.873402 22539 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:17.874110 22539 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:17.874543 22348 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:17.875523 21740 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:17.877033 22532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:17.880539 21811 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:17.881542 21811 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:59267
I20260806 07:48:17.974718 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22406
I20260806 07:48:17.991207 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35507
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--webserver_port=38941
--webserver_interface=127.20.202.193
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260806 07:48:18.089329 22172 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:18.090438 22348 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6676a466cf5b4cd8ba98d24a7d2d660e" instance_seqno: 1786002496019652) as {username='slave'} at 127.20.202.194:51401; Asking this server to re-register.
I20260806 07:48:18.091014 22172 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:18.092065 22348 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:34471)
I20260806 07:48:18.177903 22412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:18.178270 22412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: LEADER
W20260806 07:48:18.339666 22549 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:18.340129 22549 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:18.340274 22549 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:48:18.340368 22549 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:18.348999 22549 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:18.349328 22549 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:18.360138 22549 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35507
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38941
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:18.361958 22549 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:18.363832 22549 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:18.375345 22556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:18.375319 22555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:18.376284 22558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:18.376644 22549 server_base.cc:1061] running on GCE node
I20260806 07:48:18.377236 22549 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:18.377363 22549 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:18.377440 22549 hybrid_clock.cc:648] HybridClock initialized: now 1786002498377410 us; error 0 us; skew 500 ppm
I20260806 07:48:18.377799 22549 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:18.380057 22549 webserver.cc:533] Webserver started at http://127.20.202.193:38941/ using document root <none> and password file <none>
I20260806 07:48:18.380779 22549 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:18.380889 22549 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:18.387116 22549 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 07:48:18.390035 22563 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:18.391420 22549 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:48:18.391597 22549 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:18.392381 22549 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:18.426285 22549 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:18.427275 22549 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:18.427577 22549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:18.428643 22549 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:18.430454 22549 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:18.430550 22549 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:18.430711 22549 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:18.430789 22549 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:18.431209 22549 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:48:18.431336 22549 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:48:18.431435 22549 fault_injection.cc:59] injecting fault for kudu (pid 22549): FLAGS_fault_before_start (process will exit)
I20260806 07:48:18.453495 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35507
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--webserver_port=38941
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
W20260806 07:48:18.798789 22571 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:18.799180 22571 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:18.799317 22571 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:18.807827 22571 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:18.808079 22571 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:18.818310 22571 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35507
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38941
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:18.819993 22571 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:18.821857 22571 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:18.832576 22576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:18.832605 22577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:18.832690 22579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:18.834276 22571 server_base.cc:1061] running on GCE node
I20260806 07:48:18.834884 22571 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:18.835083 22571 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:18.835184 22571 hybrid_clock.cc:648] HybridClock initialized: now 1786002498835161 us; error 0 us; skew 500 ppm
I20260806 07:48:18.835529 22571 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:18.837927 22571 webserver.cc:533] Webserver started at http://127.20.202.193:38941/ using document root <none> and password file <none>
I20260806 07:48:18.838828 22571 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:18.838979 22571 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:18.844684 22571 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:48:18.847486 22584 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:18.848899 22571 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:48:18.849081 22571 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:18.849908 22571 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:18.866081 22571 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:18.867179 22571 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:18.867473 22571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:18.868446 22571 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:18.870205 22571 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:18.870307 22571 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:18.870458 22571 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:18.870538 22571 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:18.916163 22571 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35507
I20260806 07:48:18.916200 22696 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35507 every 8 connection(s)
I20260806 07:48:18.918016 22571 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:18.925050 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22571
I20260806 07:48:18.925513 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22571
I20260806 07:48:18.966418 22697 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:18.966602 22698 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:18.966898 22698 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:18.966897 22697 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:18.968235 22699 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:18.968408 22697 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:18.968539 22692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:18.968430 22699 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:18.968433 22698 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:19.062068 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22571
I20260806 07:48:19.062253 22699 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:19.062816 21811 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:19.063169 22348 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:35507)
I20260806 07:48:19.064108 21811 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:48929
W20260806 07:48:19.083174 22334 connection.cc:573] server connection from 127.20.202.193:41167 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:19.083434 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22045
W20260806 07:48:19.083165 21797 connection.cc:573] server connection from 127.20.202.193:48929 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:19.092033 22166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:19.179557 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22045
I20260806 07:48:19.197965 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22181
I20260806 07:48:19.206430 22302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:19.293881 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22181
I20260806 07:48:19.313922 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22315
I20260806 07:48:19.320607 22378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:19.430238 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22315
I20260806 07:48:19.452652 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21707
I20260806 07:48:19.453994 21771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:19.474879 21779 master_runner.cc:187] Error getting master registration for 127.20.202.254:33129: Network error: Client connection negotiation failed: client connection to 127.20.202.254:33129: connect: Connection refused (error 111),
I20260806 07:48:19.475102 21779 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:19.563799 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21707
W20260806 07:48:19.585733 21797 connection.cc:573] client connection to 127.20.202.253:45385 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:19.586151 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21779
I20260806 07:48:19.587445 21842 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:48:19.587530 21797 consensus_peers.cc:597] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf -> Peer 8bbf61a6f4e641abaf398d261b2bf14a (127.20.202.253:45385): Couldn't send request to peer 8bbf61a6f4e641abaf398d261b2bf14a. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:48:19.699442 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21779
I20260806 07:48:19.727077 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--webserver_interface=127.20.202.254
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
W20260806 07:48:20.077211 22720 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:20.077678 22720 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:20.077832 22720 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:20.087168 22720 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:20.087301 22720 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:20.087414 22720 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:20.087499 22720 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:20.097818 22720 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:20.099584 22720 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:20.101522 22720 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:20.109794 22726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:20.109880 22725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:20.110811 22728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:20.111828 22720 server_base.cc:1061] running on GCE node
I20260806 07:48:20.112604 22720 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:20.112797 22720 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:20.112893 22720 hybrid_clock.cc:648] HybridClock initialized: now 1786002500112854 us; error 0 us; skew 500 ppm
I20260806 07:48:20.113291 22720 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:20.116142 22720 webserver.cc:533] Webserver started at http://127.20.202.254:46381/ using document root <none> and password file <none>
I20260806 07:48:20.116950 22720 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:20.117079 22720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:20.123039 22720 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 07:48:20.126189 22733 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:20.127669 22720 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:48:20.127993 22720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6f79a352d4aa4102baa5b53a4388bfec"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:20.128746 22720 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:20.162323 22720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:20.163421 22720 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:20.163775 22720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:20.186928 22720 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:33129
I20260806 07:48:20.186966 22784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:33129 every 8 connection(s)
I20260806 07:48:20.188736 22720 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:20.195956 22785 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:20.198201 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22720
I20260806 07:48:20.198421 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:48:20.199905 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45385
--webserver_interface=127.20.202.253
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
I20260806 07:48:20.200033 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap starting.
I20260806 07:48:20.218451 22785 log.cc:826] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:20.227564 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:20.228101 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap complete.
I20260806 07:48:20.235214 22785 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:20.235929 22785 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f79a352d4aa4102baa5b53a4388bfec, State: Initialized, Role: FOLLOWER
I20260806 07:48:20.236620 22785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:20.237530 22789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:20.237900 22789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:20.238613 22785 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:20.248965 22800 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:20.249157 22800 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: loading cluster ID for follower catalog manager: success
I20260806 07:48:20.253463 22800 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: acquiring CA information for follower catalog manager: success
I20260806 07:48:20.256280 22800 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:20.272835 22720 master_runner.cc:187] Error getting master registration for 127.20.202.253:45385: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45385: connect: Connection refused (error 111),
I20260806 07:48:20.278776 22720 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:20.552729 22787 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:20.553062 22787 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:20.553174 22787 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:20.562132 22787 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:20.562228 22787 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:20.562317 22787 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:20.562407 22787 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:20.572734 22787 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:20.574529 22787 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:20.576413 22787 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:20.584520 22809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:20.585093 22808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:20.586072 22811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:20.587049 22787 server_base.cc:1061] running on GCE node
I20260806 07:48:20.587823 22787 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:20.588020 22787 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:20.588121 22787 hybrid_clock.cc:648] HybridClock initialized: now 1786002500588085 us; error 0 us; skew 500 ppm
I20260806 07:48:20.588477 22787 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:20.591363 22787 webserver.cc:533] Webserver started at http://127.20.202.253:41191/ using document root <none> and password file <none>
I20260806 07:48:20.592133 22787 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:20.592279 22787 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:20.597961 22787 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:48:20.600984 22816 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:20.602478 22787 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:20.602658 22787 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "8bbf61a6f4e641abaf398d261b2bf14a"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:20.603369 22787 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:20.627064 22787 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:20.628093 22787 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:20.628427 22787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:20.651676 22867 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:45385 every 8 connection(s)
I20260806 07:48:20.651674 22787 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:45385
I20260806 07:48:20.653476 22787 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:48:20.661078 22868 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:20.662011 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22787
I20260806 07:48:20.662245 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:48:20.663235 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36587
--webserver_interface=127.20.202.252
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
I20260806 07:48:20.664724 22868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Bootstrap starting.
I20260806 07:48:20.686477 22868 log.cc:826] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:20.695684 22868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:20.696245 22868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Bootstrap complete.
I20260806 07:48:20.703081 22868 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:20.703785 22868 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbf61a6f4e641abaf398d261b2bf14a, State: Initialized, Role: FOLLOWER
I20260806 07:48:20.704487 22868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:20.705407 22872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:20.705802 22872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:20.706372 22868 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:20.717106 22883 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:20.717290 22883 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: loading cluster ID for follower catalog manager: success
I20260806 07:48:20.722558 22883 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: acquiring CA information for follower catalog manager: success
I20260806 07:48:20.725363 22883 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:20.754120 22787 master_runner.cc:187] Error getting master registration for 127.20.202.252:36587: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36587: connect: Connection refused (error 111),
I20260806 07:48:20.760463 22787 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:21.027004 22870 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:21.027381 22870 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:21.027523 22870 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:21.036325 22870 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:21.036466 22870 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:21.036586 22870 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:21.036669 22870 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:21.047382 22870 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:21.049188 22870 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:21.051151 22870 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:21.060185 22891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:21.061327 22892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:21.062130 22894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:21.063186 22870 server_base.cc:1061] running on GCE node
I20260806 07:48:21.064191 22870 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:21.064399 22870 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:21.064502 22870 hybrid_clock.cc:648] HybridClock initialized: now 1786002501064478 us; error 0 us; skew 500 ppm
I20260806 07:48:21.064975 22870 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:21.068722 22870 webserver.cc:533] Webserver started at http://127.20.202.252:46409/ using document root <none> and password file <none>
I20260806 07:48:21.069793 22870 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:21.069952 22870 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:21.077093 22870 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:48:21.080608 22899 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:21.082376 22870 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:21.082548 22870 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf"
format_stamp: "Formatted at 2026-08-06 07:48:14 on dist-test-slave-g350"
I20260806 07:48:21.083499 22870 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:21.108821 22870 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:21.110102 22870 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:21.110425 22870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:21.134258 22870 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:36587
I20260806 07:48:21.134276 22950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:36587 every 8 connection(s)
I20260806 07:48:21.136409 22870 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:48:21.137001 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22870
I20260806 07:48:21.137209 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:48:21.149160 22951 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:21.153669 22951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Bootstrap starting.
I20260806 07:48:21.175988 22951 log.cc:826] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:21.186692 22951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:21.187296 22951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: Bootstrap complete.
I20260806 07:48:21.194732 22951 raft_consensus.cc:359] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:21.195796 22951 raft_consensus.cc:740] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ae6bbd5d2b4088b7e7338ef4fae5bf, State: Initialized, Role: FOLLOWER
I20260806 07:48:21.196446 22951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:21.197325 22956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:21.197677 22956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:21.198493 22951 sys_catalog.cc:565] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:21.210198 22967 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:21.210314 22967 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: loading cluster ID for follower catalog manager: success
I20260806 07:48:21.215099 22967 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: acquiring CA information for follower catalog manager: success
I20260806 07:48:21.217918 22967 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:21.237425 22870 master_runner.cc:264] No leader master found from master 32ae6bbd5d2b4088b7e7338ef4fae5bf
I20260806 07:48:21.237591 22870 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:21.722805 22970 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:48:21.723106 22970 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:21.724990 22970 leader_election.cc:290] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bbf61a6f4e641abaf398d261b2bf14a (127.20.202.253:45385), 32ae6bbd5d2b4088b7e7338ef4fae5bf (127.20.202.252:36587)
I20260806 07:48:21.731684 22843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f79a352d4aa4102baa5b53a4388bfec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" is_pre_election: true
I20260806 07:48:21.732291 22843 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f79a352d4aa4102baa5b53a4388bfec in term 1.
I20260806 07:48:21.732360 22926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f79a352d4aa4102baa5b53a4388bfec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" is_pre_election: true
I20260806 07:48:21.732916 22926 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f79a352d4aa4102baa5b53a4388bfec in term 1.
I20260806 07:48:21.733183 22735 leader_election.cc:304] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f79a352d4aa4102baa5b53a4388bfec, 8bbf61a6f4e641abaf398d261b2bf14a; no voters:
I20260806 07:48:21.733700 22970 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:48:21.733871 22970 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:48:21.733999 22970 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:21.737810 22970 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:21.738484 22970 leader_election.cc:290] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [CANDIDATE]: Term 2 election: Requested vote from peers 8bbf61a6f4e641abaf398d261b2bf14a (127.20.202.253:45385), 32ae6bbd5d2b4088b7e7338ef4fae5bf (127.20.202.252:36587)
I20260806 07:48:21.738988 22926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f79a352d4aa4102baa5b53a4388bfec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf"
I20260806 07:48:21.738999 22843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f79a352d4aa4102baa5b53a4388bfec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8bbf61a6f4e641abaf398d261b2bf14a"
I20260806 07:48:21.739207 22926 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:21.739214 22843 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:21.743049 22843 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f79a352d4aa4102baa5b53a4388bfec in term 2.
I20260806 07:48:21.743048 22926 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f79a352d4aa4102baa5b53a4388bfec in term 2.
I20260806 07:48:21.743573 22735 leader_election.cc:304] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f79a352d4aa4102baa5b53a4388bfec, 8bbf61a6f4e641abaf398d261b2bf14a; no voters:
I20260806 07:48:21.743947 22970 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:48:21.744349 22970 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 LEADER]: Becoming Leader. State: Replica: 6f79a352d4aa4102baa5b53a4388bfec, State: Running, Role: LEADER
I20260806 07:48:21.744817 22970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:21.747752 22973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f79a352d4aa4102baa5b53a4388bfec. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:21.748132 22973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:21.748840 22975 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:21.750864 22975 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:21.751675 22975 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:21.751745 22975 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:21.752820 22975 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:21.753700 22975 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Loaded TSK: 0
I20260806 07:48:21.754611 22975 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:21.769491 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--time_source=system_unsync with env {}
W20260806 07:48:22.124507 22976 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:22.124814 22976 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:22.124969 22976 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:22.133292 22976 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:22.133520 22976 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:22.144169 22843 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Refusing update from remote peer 6f79a352d4aa4102baa5b53a4388bfec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:48:22.144345 22976 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:22.145262 22973 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:48:22.146329 22976 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:22.148159 22976 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:48:22.149032 22980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f79a352d4aa4102baa5b53a4388bfec. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:22.149366 22980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:22.151031 22970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:22.151328 22970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:22.152714 22926 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 2 FOLLOWER]: Refusing update from remote peer 6f79a352d4aa4102baa5b53a4388bfec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:48:22.152911 22980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:22.153115 22980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:22.153613 22983 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:48:22.158845 22990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:22.159241 22990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:22.159786 22985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f79a352d4aa4102baa5b53a4388bfec. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:22.160183 22985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:48:22.160838 22989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:22.161348 22991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:22.165983 22976 server_base.cc:1061] running on GCE node
W20260806 07:48:22.166265 22993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:22.166942 22976 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:22.167142 22976 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:22.167244 22976 hybrid_clock.cc:648] HybridClock initialized: now 1786002502167221 us; error 0 us; skew 500 ppm
I20260806 07:48:22.167585 22976 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:22.170094 22976 webserver.cc:533] Webserver started at http://127.20.202.193:39835/ using document root <none> and password file <none>
I20260806 07:48:22.170944 22976 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:22.171098 22976 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:22.176687 22976 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:48:22.179751 22998 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:22.181195 22976 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:22.181409 22976 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:22.182225 22976 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:22.199012 22976 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:22.199965 22976 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:22.200277 22976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:22.201282 22976 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:22.203205 22976 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:22.203310 22976 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:22.203461 22976 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:22.203531 22976 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:22.238132 22976 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36761
I20260806 07:48:22.238222 23110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36761 every 8 connection(s)
I20260806 07:48:22.239768 22976 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:22.240512 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22976
I20260806 07:48:22.240795 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:48:22.256767 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--time_source=system_unsync with env {}
I20260806 07:48:22.277460 23111 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:22.277488 23113 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:22.277544 23112 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:22.277972 23112 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:22.277968 23111 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:22.277968 23113 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:22.279371 23113 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:22.279351 23111 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:22.279347 23112 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:22.282033 22916 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
I20260806 07:48:22.282063 22750 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
I20260806 07:48:22.282708 22833 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
I20260806 07:48:22.284030 22750 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:53825
W20260806 07:48:22.605175 23121 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:22.605551 23121 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:22.605710 23121 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:22.614379 23121 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:22.614611 23121 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:48:22.625149 23121 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:22.626910 23121 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:22.628803 23121 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:22.638604 23127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:22.638794 23126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:22.639000 23129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:22.640422 23121 server_base.cc:1061] running on GCE node
I20260806 07:48:22.640961 23121 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:22.641115 23121 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:22.641182 23121 hybrid_clock.cc:648] HybridClock initialized: now 1786002502641166 us; error 0 us; skew 500 ppm
I20260806 07:48:22.641434 23121 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:22.643776 23121 webserver.cc:533] Webserver started at http://127.20.202.194:42897/ using document root <none> and password file <none>
I20260806 07:48:22.644528 23121 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:22.644645 23121 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:22.650679 23121 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:48:22.653861 23134 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:22.655352 23121 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:22.655512 23121 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "6676a466cf5b4cd8ba98d24a7d2d660e"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:22.656243 23121 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:22.677561 23121 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:22.678547 23121 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:22.678869 23121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:22.679862 23121 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:22.681568 23121 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:22.681664 23121 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:48:22.681833 23121 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:22.681885 23121 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:22.717765 23121 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:43955
I20260806 07:48:22.717856 23246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:43955 every 8 connection(s)
I20260806 07:48:22.719413 23121 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:48:22.724432 22983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6f79a352d4aa4102baa5b53a4388bfec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:22.724704 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23121
I20260806 07:48:22.724714 22983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:22.724879 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:48:22.733724 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--time_source=system_unsync with env {}
I20260806 07:48:22.771073 23248 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:22.771165 23249 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:22.771183 23247 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:22.771569 23248 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:22.771557 23247 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:22.771559 23249 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:22.772758 23248 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:22.772871 23249 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:22.773993 23247 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:22.774484 22750 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:43955)
I20260806 07:48:22.774729 22833 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:43955)
I20260806 07:48:22.775183 22916 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:43955)
I20260806 07:48:22.775596 22750 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:50687
W20260806 07:48:23.093534 23257 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:23.093924 23257 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:23.094007 23257 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:23.102349 23257 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:23.102511 23257 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:48:23.112434 23257 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:23.114109 23257 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:23.116092 23257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:23.125716 23262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:23.125753 23263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:23.130618 23265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:23.131140 23257 server_base.cc:1061] running on GCE node
I20260806 07:48:23.131676 23257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:23.131817 23257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:23.131882 23257 hybrid_clock.cc:648] HybridClock initialized: now 1786002503131864 us; error 0 us; skew 500 ppm
I20260806 07:48:23.132117 23257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:23.134661 23257 webserver.cc:533] Webserver started at http://127.20.202.195:44595/ using document root <none> and password file <none>
I20260806 07:48:23.135630 23257 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:23.135779 23257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:23.143147 23257 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 07:48:23.146486 23270 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:23.148161 23257 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:48:23.148340 23257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "f197f2ce7d9e4359bfbf6ee094135742"
format_stamp: "Formatted at 2026-08-06 07:48:16 on dist-test-slave-g350"
I20260806 07:48:23.149070 23257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:23.172299 23257 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:23.173358 23257 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:23.173727 23257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:23.174844 23257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:23.176724 23257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:23.176808 23257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:23.177017 23257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:23.177093 23257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:23.215207 23257 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:46023
I20260806 07:48:23.215243 23382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:46023 every 8 connection(s)
I20260806 07:48:23.216835 23257 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:48:23.221400 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23257
I20260806 07:48:23.221695 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:48:23.241506 23385 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:23.241978 23385 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:23.243562 23385 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:23.244539 23384 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:23.244665 23383 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:23.244710 23384 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:23.244830 23383 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:23.245379 23384 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:23.246097 23383 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:23.246318 22750 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:46023)
I20260806 07:48:23.246412 22833 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:46023)
I20260806 07:48:23.247431 22750 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:47883
I20260806 07:48:23.247913 22916 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:46023)
I20260806 07:48:23.266145 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:23.274427 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:23.274672 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22720
I20260806 07:48:23.275610 22780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:23.286513 23112 heartbeater.cc:499] Master 127.20.202.254:33129 was elected leader, sending a full tablet report...
I20260806 07:48:23.387173 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22720
W20260806 07:48:23.410516 22900 connection.cc:573] server connection from 127.0.0.1:37164 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:23.411324 23112 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:33129 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.20.202.254:33129 (error 108)
I20260806 07:48:23.412288 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--webserver_interface=127.20.202.254
--webserver_port=46381
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587 with env {}
W20260806 07:48:23.759697 23391 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:23.760100 23391 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:23.760236 23391 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:23.769475 23391 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:23.769603 23391 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:23.769744 23391 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:23.769832 23391 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:23.778239 23248 heartbeater.cc:499] Master 127.20.202.254:33129 was elected leader, sending a full tablet report...
W20260806 07:48:23.779909 23248 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:33129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:33129: connect: Connection refused (error 111)
I20260806 07:48:23.781674 23391 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=46381
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:23.783356 23391 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:23.785240 23391 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:23.793833 23398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:23.793799 23397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:23.794601 23400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:23.795467 23391 server_base.cc:1061] running on GCE node
I20260806 07:48:23.796226 23391 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:23.796413 23391 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:23.796516 23391 hybrid_clock.cc:648] HybridClock initialized: now 1786002503796478 us; error 0 us; skew 500 ppm
I20260806 07:48:23.796875 23391 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:23.799773 23391 webserver.cc:533] Webserver started at http://127.20.202.254:46381/ using document root <none> and password file <none>
I20260806 07:48:23.800704 23391 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:23.800794 23391 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:23.806516 23391 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:48:23.809644 23405 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:23.811065 23391 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:48:23.811283 23391 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6f79a352d4aa4102baa5b53a4388bfec"
format_stamp: "Formatted at 2026-08-06 07:48:13 on dist-test-slave-g350"
I20260806 07:48:23.812297 23391 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:23.831516 23391 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:23.832471 23391 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:23.832783 23391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:23.855432 23391 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:33129
I20260806 07:48:23.855499 23456 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:33129 every 8 connection(s)
I20260806 07:48:23.857196 23391 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:23.862880 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23391
I20260806 07:48:23.863276 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22976
I20260806 07:48:23.865446 23457 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:23.868891 23457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap starting.
I20260806 07:48:23.873169 23106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:23.893177 23457 log.cc:826] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:23.903601 23457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:23.904143 23457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: Bootstrap complete.
I20260806 07:48:23.911664 23457 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:23.912375 23457 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f79a352d4aa4102baa5b53a4388bfec, State: Initialized, Role: FOLLOWER
I20260806 07:48:23.913193 23457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:23.914150 23463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:23.914479 23463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:23.915092 23457 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:23.937363 23474 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:23.937516 23474 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: loading cluster ID for follower catalog manager: success
I20260806 07:48:23.942097 23474 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: acquiring CA information for follower catalog manager: success
I20260806 07:48:23.944854 23474 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:23.977567 23391 master_runner.cc:264] No leader master found from master 6f79a352d4aa4102baa5b53a4388bfec
I20260806 07:48:23.977797 23391 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:24.007941 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22976
I20260806 07:48:24.033885 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36761
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--webserver_port=39835
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
I20260806 07:48:24.250327 23384 heartbeater.cc:499] Master 127.20.202.254:33129 was elected leader, sending a full tablet report...
I20260806 07:48:24.262633 23422 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "f197f2ce7d9e4359bfbf6ee094135742" instance_seqno: 1786002503206218) as {username='slave'} at 127.20.202.195:36489; Asking this server to re-register.
I20260806 07:48:24.263797 23384 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:24.265937 23422 ts_manager.cc:194] Registered new tserver with Master: f197f2ce7d9e4359bfbf6ee094135742 (127.20.202.195:46023)
W20260806 07:48:24.413223 23480 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:24.413921 23480 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:24.414084 23480 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:24.423463 23480 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:24.423741 23480 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:24.435170 23480 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36761
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=39835
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:24.437029 23480 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:24.439076 23480 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:24.450562 23486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:24.450541 23487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:24.453824 23489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:24.454145 23480 server_base.cc:1061] running on GCE node
I20260806 07:48:24.455014 23480 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:24.455170 23480 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:24.455245 23480 hybrid_clock.cc:648] HybridClock initialized: now 1786002504455222 us; error 0 us; skew 500 ppm
I20260806 07:48:24.455684 23480 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:24.458354 23480 webserver.cc:533] Webserver started at http://127.20.202.193:39835/ using document root <none> and password file <none>
I20260806 07:48:24.459173 23480 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:24.459281 23480 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:24.465932 23480 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 07:48:24.468952 23494 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:24.470654 23480 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:24.470850 23480 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:24.471668 23480 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:24.485797 23480 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:24.486781 23480 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:24.487196 23480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:24.488272 23480 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:24.490362 23480 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:24.490490 23480 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:24.490656 23480 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:24.490741 23480 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:24.505890 23535 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6f79a352d4aa4102baa5b53a4388bfec)
I20260806 07:48:24.506116 23535 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:24.508178 23535 leader_election.cc:290] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f79a352d4aa4102baa5b53a4388bfec (127.20.202.254:33129), 32ae6bbd5d2b4088b7e7338ef4fae5bf (127.20.202.252:36587)
I20260806 07:48:24.518465 22926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" is_pre_election: true
I20260806 07:48:24.518514 23432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6f79a352d4aa4102baa5b53a4388bfec" is_pre_election: true
I20260806 07:48:24.519271 23432 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bbf61a6f4e641abaf398d261b2bf14a in term 2.
I20260806 07:48:24.520031 22817 leader_election.cc:304] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f79a352d4aa4102baa5b53a4388bfec, 8bbf61a6f4e641abaf398d261b2bf14a; no voters: 32ae6bbd5d2b4088b7e7338ef4fae5bf
I20260806 07:48:24.520535 23535 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:48:24.520664 23535 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6f79a352d4aa4102baa5b53a4388bfec)
I20260806 07:48:24.520763 23535 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:48:24.523649 23535 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:24.524802 23535 leader_election.cc:290] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [CANDIDATE]: Term 3 election: Requested vote from peers 6f79a352d4aa4102baa5b53a4388bfec (127.20.202.254:33129), 32ae6bbd5d2b4088b7e7338ef4fae5bf (127.20.202.252:36587)
I20260806 07:48:24.524896 22926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf"
I20260806 07:48:24.524952 23432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6f79a352d4aa4102baa5b53a4388bfec"
I20260806 07:48:24.525277 23432 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:48:24.532408 23432 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bbf61a6f4e641abaf398d261b2bf14a in term 3.
I20260806 07:48:24.533175 22817 leader_election.cc:304] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f79a352d4aa4102baa5b53a4388bfec, 8bbf61a6f4e641abaf398d261b2bf14a; no voters: 32ae6bbd5d2b4088b7e7338ef4fae5bf
I20260806 07:48:24.533641 23535 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:48:24.534186 23535 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [term 3 LEADER]: Becoming Leader. State: Replica: 8bbf61a6f4e641abaf398d261b2bf14a, State: Running, Role: LEADER
I20260806 07:48:24.534749 23535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:24.536753 23593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bbf61a6f4e641abaf398d261b2bf14a. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:24.537111 23593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:24.538010 23600 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:24.541306 23600 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:24.542841 23600 catalog_manager.cc:1295] Loaded cluster ID: 831d8a18222040cab1f6d5464da1cc71
I20260806 07:48:24.542960 23600 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:24.544157 23613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36761 every 8 connection(s)
I20260806 07:48:24.544157 23480 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36761
I20260806 07:48:24.544759 23600 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:24.545957 23600 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a: Loaded TSK: 0
I20260806 07:48:24.546555 23480 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:24.547209 23600 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:24.548806 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23480
I20260806 07:48:24.549178 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23480
I20260806 07:48:24.574743 23614 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:24.575261 23614 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:24.575645 23609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:24.663151 23614 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:24.665297 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23480
I20260806 07:48:24.665664 22916 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
W20260806 07:48:24.684288 22901 connection.cc:573] server connection from 127.20.202.193:47993 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:24.684368 23406 connection.cc:573] server connection from 127.20.202.193:37919 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:24.685479 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36761
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--webserver_port=39835
--webserver_interface=127.20.202.193
--time_source=system_unsync
--fault_before_start=1.0 with env {}
W20260806 07:48:24.685731 23622 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:48:24.558116 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:37281 (local address 127.20.202.253:45385)
0806 07:48:24.577583 (+ 19467us) server_negotiation.cc:207] Beginning negotiation
0806 07:48:24.577607 (+ 24us) server_negotiation.cc:400] Waiting for connection header
0806 07:48:24.577633 (+ 26us) server_negotiation.cc:408] Connection header received
0806 07:48:24.577702 (+ 69us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:48:24.577709 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:48:24.577803 (+ 94us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:48:24.577946 (+ 143us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:48:24.684589 (+106643us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:37281: BlockingRecv error: recv got EOF from 127.20.202.193:37281 (error 108)
Metrics: {"server-negotiator.queue_time_us":19321,"thread_start_us":20087,"threads_started":1}
I20260806 07:48:24.788453 23248 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:24.788700 23248 heartbeater.cc:499] Master 127.20.202.254:33129 was elected leader, sending a full tablet report...
I20260806 07:48:24.789908 23422 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6676a466cf5b4cd8ba98d24a7d2d660e" instance_seqno: 1786002502709006) as {username='slave'} at 127.20.202.194:34287; Asking this server to re-register.
I20260806 07:48:24.790652 23248 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:24.791869 23422 ts_manager.cc:194] Registered new tserver with Master: 6676a466cf5b4cd8ba98d24a7d2d660e (127.20.202.194:43955)
I20260806 07:48:24.888603 23627 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6f79a352d4aa4102baa5b53a4388bfec)
I20260806 07:48:24.888880 23627 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } }
I20260806 07:48:24.890921 23627 leader_election.cc:290] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f79a352d4aa4102baa5b53a4388bfec (127.20.202.254:33129), 8bbf61a6f4e641abaf398d261b2bf14a (127.20.202.253:45385)
I20260806 07:48:24.892055 22843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" is_pre_election: true
I20260806 07:48:24.898442 23432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6f79a352d4aa4102baa5b53a4388bfec" is_pre_election: true
I20260806 07:48:24.898945 23432 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32ae6bbd5d2b4088b7e7338ef4fae5bf in current term 3: Already voted for candidate 8bbf61a6f4e641abaf398d261b2bf14a in this term.
I20260806 07:48:24.899739 22900 leader_election.cc:304] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32ae6bbd5d2b4088b7e7338ef4fae5bf; no voters: 6f79a352d4aa4102baa5b53a4388bfec, 8bbf61a6f4e641abaf398d261b2bf14a
I20260806 07:48:24.900230 23627 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:48:24.903342 23627 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260806 07:48:25.037334 23625 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:25.037721 23625 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:25.037866 23625 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:48:25.037971 23625 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:25.046231 23625 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:25.046459 23625 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:25.056841 23625 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36761
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=39835
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:25.058665 23625 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:25.060550 23625 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:25.071061 23634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:25.070977 23636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:25.071014 23633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:25.072307 23625 server_base.cc:1061] running on GCE node
I20260806 07:48:25.072958 23625 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:25.073168 23625 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:25.073300 23625 hybrid_clock.cc:648] HybridClock initialized: now 1786002505073250 us; error 0 us; skew 500 ppm
I20260806 07:48:25.073727 23625 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:25.076262 23625 webserver.cc:533] Webserver started at http://127.20.202.193:39835/ using document root <none> and password file <none>
I20260806 07:48:25.076900 23432 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [term 3 FOLLOWER]: Refusing update from remote peer 8bbf61a6f4e641abaf398d261b2bf14a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:48:25.077068 23625 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:25.077236 23625 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:25.077879 23638 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:48:25.081529 23641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bbf61a6f4e641abaf398d261b2bf14a. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:25.081885 23641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:25.083266 23638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:25.083580 23638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:25.084942 22926 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [term 3 FOLLOWER]: Refusing update from remote peer 8bbf61a6f4e641abaf398d261b2bf14a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:48:25.096410 23638 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260806 07:48:25.096472 23641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:25.096711 23641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f79a352d4aa4102baa5b53a4388bfec [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:25.098042 23625 fs_manager.cc:714] Time spent opening directory manager: real 0.018s user 0.009s sys 0.009s
I20260806 07:48:25.099701 23627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bbf61a6f4e641abaf398d261b2bf14a. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:25.099917 23627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:25.099630 23651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:25.100199 23651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ae6bbd5d2b4088b7e7338ef4fae5bf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:25.101823 23652 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:25.103343 23625 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:25.103500 23625 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:25.104290 23625 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:25.122278 23625 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:25.123131 23625 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:25.123378 23625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:25.124262 23625 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:25.126005 23625 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:25.126094 23625 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:25.126178 23625 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:25.126231 23625 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:25.126556 23625 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:48:25.126677 23625 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:48:25.126773 23625 fault_injection.cc:59] injecting fault for kudu (pid 23625): FLAGS_fault_before_start (process will exit)
I20260806 07:48:25.146694 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36761
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--webserver_port=39835
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
W20260806 07:48:25.490839 23660 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:25.491218 23660 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:25.491304 23660 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:25.499773 23660 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:25.499933 23660 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:25.510222 23660 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36761
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=39835
--tserver_master_addrs=127.20.202.254:33129,127.20.202.253:45385,127.20.202.252:36587
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:25.511862 23660 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:25.513741 23660 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:25.523533 23666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:25.523589 23665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:25.524317 23668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:25.524874 23660 server_base.cc:1061] running on GCE node
I20260806 07:48:25.525462 23660 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:48:25.525684 23660 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:48:25.525785 23660 hybrid_clock.cc:648] HybridClock initialized: now 1786002505525764 us; error 0 us; skew 500 ppm
I20260806 07:48:25.526093 23660 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:25.528183 23660 webserver.cc:533] Webserver started at http://127.20.202.193:39835/ using document root <none> and password file <none>
I20260806 07:48:25.528929 23660 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:25.529078 23660 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:25.534545 23660 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:48:25.537173 23673 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:25.538765 23660 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:25.538959 23660 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "667f53789cc84714be81ac7874b2e274"
format_stamp: "Formatted at 2026-08-06 07:48:15 on dist-test-slave-g350"
I20260806 07:48:25.539846 23660 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:25.561874 23660 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:25.562837 23660 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:25.563104 23660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:25.564072 23660 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:25.565779 23660 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:25.565853 23660 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:25.565968 23660 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:25.566036 23660 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:25.600919 23660 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36761
I20260806 07:48:25.600950 23785 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36761 every 8 connection(s)
I20260806 07:48:25.602638 23660 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:25.607199 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23660
I20260806 07:48:25.607604 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23660
I20260806 07:48:25.626175 23788 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45385
I20260806 07:48:25.626571 23788 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:25.627416 23788 heartbeater.cc:507] Master 127.20.202.253:45385 requested a full tablet report, sending...
I20260806 07:48:25.629289 22833 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
I20260806 07:48:25.631067 22833 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:46417
I20260806 07:48:25.634066 23786 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36587
I20260806 07:48:25.634227 23786 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:25.634590 23786 heartbeater.cc:507] Master 127.20.202.252:36587 requested a full tablet report, sending...
I20260806 07:48:25.635757 22916 ts_manager.cc:194] Re-registered known tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
I20260806 07:48:25.637382 23787 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33129
I20260806 07:48:25.637513 23787 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:25.637693 23781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:25.662977 23646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f79a352d4aa4102baa5b53a4388bfec" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33129 } } peers { permanent_uuid: "8bbf61a6f4e641abaf398d261b2bf14a" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45385 } } peers { permanent_uuid: "32ae6bbd5d2b4088b7e7338ef4fae5bf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36587 } } }
I20260806 07:48:25.663319 23646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbf61a6f4e641abaf398d261b2bf14a [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:25.726553 23787 heartbeater.cc:507] Master 127.20.202.254:33129 requested a full tablet report, sending...
I20260806 07:48:25.728482 23422 ts_manager.cc:194] Registered new tserver with Master: 667f53789cc84714be81ac7874b2e274 (127.20.202.193:36761)
I20260806 07:48:25.729072 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23660
I20260806 07:48:25.749022 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23121
I20260806 07:48:25.758262 23242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:25.845763 23249 heartbeater.cc:499] Master 127.20.202.253:45385 was elected leader, sending a full tablet report...
I20260806 07:48:25.847510 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23121
W20260806 07:48:25.870864 23406 connection.cc:573] server connection from 127.20.202.194:34287 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:25.871606 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23257
I20260806 07:48:25.880671 23378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:25.970873 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23257
I20260806 07:48:25.989128 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23391
I20260806 07:48:25.996132 23452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:26.103618 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23391
I20260806 07:48:26.126783 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22787
I20260806 07:48:26.128080 22863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:26.240543 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22787
I20260806 07:48:26.240137 22870 master_runner.cc:187] Error getting master registration for 127.20.202.254:33129: Network error: Client connection negotiation failed: client connection to 127.20.202.254:33129: connect: Connection refused (error 111),
I20260806 07:48:26.246609 22870 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:26.265524 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22870
I20260806 07:48:26.266829 22946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:26.374380 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22870
[ OK ] ExternalMiniClusterTest.TestBasicOperation/0 (13432 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/1
2026-08-06T07:48:26Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:26Z Disabled control of system clock
I20260806 07:48:26.436622 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
W20260806 07:48:26.776890 23813 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:26.777212 23813 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:26.777356 23813 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:26.785792 23813 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:26.785900 23813 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:26.785984 23813 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:26.786055 23813 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:26.796362 23813 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:26.798112 23813 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:26.800043 23813 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:26.808349 23819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:26.808301 23818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:26.809016 23821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:26.809769 23813 server_base.cc:1061] running on GCE node
I20260806 07:48:26.810563 23813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:26.812103 23813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:26.813314 23813 hybrid_clock.cc:648] HybridClock initialized: now 1786002506813261 us; error 39 us; skew 500 ppm
I20260806 07:48:26.813789 23813 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:26.816236 23813 webserver.cc:533] Webserver started at http://127.20.202.254:34511/ using document root <none> and password file <none>
I20260806 07:48:26.817044 23813 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:26.817183 23813 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:26.817643 23813 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:26.821748 23813 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
format_stamp: "Formatted at 2026-08-06 07:48:26 on dist-test-slave-g350"
I20260806 07:48:26.822669 23813 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
format_stamp: "Formatted at 2026-08-06 07:48:26 on dist-test-slave-g350"
I20260806 07:48:26.828572 23813 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260806 07:48:26.832218 23827 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:26.833740 23813 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:26.833935 23813 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
format_stamp: "Formatted at 2026-08-06 07:48:26 on dist-test-slave-g350"
I20260806 07:48:26.834172 23813 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:26.854151 23813 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:26.855105 23813 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:26.855405 23813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:26.876544 23878 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39675 every 8 connection(s)
I20260806 07:48:26.876597 23813 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39675
I20260806 07:48:26.878233 23813 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:26.882669 23879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:26.887195 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23813
I20260806 07:48:26.887411 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:48:26.888443 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45855
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
I20260806 07:48:26.888782 23879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:26.905947 23879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:26.907186 23828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111)
W20260806 07:48:26.909706 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45855: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111)
I20260806 07:48:26.959120 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } attempt: 1
W20260806 07:48:26.961373 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45855: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111)
I20260806 07:48:27.029657 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } attempt: 2
W20260806 07:48:27.031853 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45855: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111)
I20260806 07:48:27.123128 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } attempt: 3
W20260806 07:48:27.125386 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45855: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111)
W20260806 07:48:27.235654 23883 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:27.236042 23883 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:27.236182 23883 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:27.244751 23883 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:27.244884 23883 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:27.245025 23883 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:27.245112 23883 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:27.255518 23883 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:27.257292 23883 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:27.259270 23883 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:27.267840 23889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:27.267815 23890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:27.268712 23893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:27.268611 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } attempt: 4
I20260806 07:48:27.269645 23883 server_base.cc:1061] running on GCE node
I20260806 07:48:27.270769 23883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260806 07:48:27.270952 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45855: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111)
I20260806 07:48:27.272270 23883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:27.273505 23883 hybrid_clock.cc:648] HybridClock initialized: now 1786002507273441 us; error 58 us; skew 500 ppm
I20260806 07:48:27.273880 23883 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:27.276168 23883 webserver.cc:533] Webserver started at http://127.20.202.253:42073/ using document root <none> and password file <none>
I20260806 07:48:27.276984 23883 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:27.277129 23883 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:27.277534 23883 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:27.280050 23883 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:27.280894 23883 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:27.286851 23883 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260806 07:48:27.290514 23899 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:27.291968 23883 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:48:27.292177 23883 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:27.292424 23883 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:27.313052 23883 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:27.314096 23883 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:27.314396 23883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:27.335444 23883 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:45855
I20260806 07:48:27.335479 23950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:45855 every 8 connection(s)
I20260806 07:48:27.337128 23883 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:48:27.341356 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23883
I20260806 07:48:27.341570 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:48:27.341926 23951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:27.342705 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33895
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
I20260806 07:48:27.347932 23951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:27.367300 23951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:27.377153 23951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:27.378366 23900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
W20260806 07:48:27.380060 23951 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.429344 23951 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 1
W20260806 07:48:27.431636 23951 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.499853 23951 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 2
W20260806 07:48:27.502179 23951 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.570254 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } attempt: 5
I20260806 07:48:27.577489 23879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:27.579380 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.593420 23951 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 3
W20260806 07:48:27.595638 23951 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.630622 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 1
W20260806 07:48:27.632659 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
W20260806 07:48:27.688823 23953 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:27.689148 23953 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:27.689221 23953 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:27.697572 23953 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:27.697731 23953 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:27.697841 23953 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:27.697923 23953 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:27.700932 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 2
W20260806 07:48:27.703153 23879 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.708534 23953 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:27.710362 23953 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:27.712284 23953 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:27.720999 23961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:27.721287 23964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:27.721323 23962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:27.722580 23953 server_base.cc:1061] running on GCE node
I20260806 07:48:27.723315 23953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:27.724694 23953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:27.725935 23953 hybrid_clock.cc:648] HybridClock initialized: now 1786002507725877 us; error 76 us; skew 500 ppm
I20260806 07:48:27.726289 23953 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:27.728586 23953 webserver.cc:533] Webserver started at http://127.20.202.252:45327/ using document root <none> and password file <none>
I20260806 07:48:27.729329 23953 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:27.729480 23953 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:27.729877 23953 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:27.732231 23953 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "c91518017b4e465fa782aa91e17b8de9"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:27.733052 23953 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "c91518017b4e465fa782aa91e17b8de9"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:27.738513 23953 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 07:48:27.738907 23951 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 4
W20260806 07:48:27.741293 23951 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111)
I20260806 07:48:27.742856 23970 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:27.744418 23953 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 07:48:27.744654 23953 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "c91518017b4e465fa782aa91e17b8de9"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:27.744890 23953 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:27.757725 23953 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:27.758644 23953 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:27.758932 23953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:27.779848 23953 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:33895
I20260806 07:48:27.779873 24022 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:33895 every 8 connection(s)
I20260806 07:48:27.781461 23953 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:48:27.784013 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23953
I20260806 07:48:27.784238 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:48:27.786375 24023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:27.794766 24023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:27.808177 24023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:27.809357 23879 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 3
I20260806 07:48:27.815191 24023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:27.821076 23879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap starting.
I20260806 07:48:27.825320 23879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:27.826337 24023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Bootstrap starting.
I20260806 07:48:27.826581 23879 log.cc:826] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:27.829043 23879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: No bootstrap required, opened a new log
I20260806 07:48:27.830109 24023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:27.831607 24023 log.cc:826] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:27.834314 24023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: No bootstrap required, opened a new log
I20260806 07:48:27.835425 23879 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.835743 23879 raft_consensus.cc:385] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:27.835887 23879 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91b50fc26f949c4bcbf1c708b3e4328, State: Initialized, Role: FOLLOWER
I20260806 07:48:27.836673 23879 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.837708 24030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:27.837998 24030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:27.838863 23879 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:27.840411 24023 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.840708 24023 raft_consensus.cc:385] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:27.840881 24023 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91518017b4e465fa782aa91e17b8de9, State: Initialized, Role: FOLLOWER
I20260806 07:48:27.841529 24023 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.842594 24035 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:27.842905 24035 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:27.843670 24023 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:48:27.853354 24047 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:27.853569 24047 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:48:27.857918 24054 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:27.858173 24054 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:48:27.858734 23833 tablet.cc:2406] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:48:27.852234 23813 master_runner.cc:187] Error getting master registration for 127.20.202.253:45855: OK, instance_id { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" instance_seqno: 1786002507329719 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:48:27.858961 23813 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:27.863227 23953 master_runner.cc:187] Error getting master registration for 127.20.202.253:45855: OK, instance_id { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" instance_seqno: 1786002507329719 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:48:27.870090 23953 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:27.953990 24030 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:48:27.954300 24030 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.956239 24030 leader_election.cc:290] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855), c91518017b4e465fa782aa91e17b8de9 (127.20.202.252:33895)
I20260806 07:48:27.957202 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91518017b4e465fa782aa91e17b8de9" is_pre_election: true
I20260806 07:48:27.957826 23998 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 0.
I20260806 07:48:27.958500 23828 leader_election.cc:304] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328; no voters:
I20260806 07:48:27.958942 24030 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:48:27.959100 24030 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:48:27.959213 24030 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:27.962744 24030 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.963500 24030 leader_election.cc:290] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 1 election: Requested vote from peers a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855), c91518017b4e465fa782aa91e17b8de9 (127.20.202.252:33895)
I20260806 07:48:27.964293 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91518017b4e465fa782aa91e17b8de9"
I20260806 07:48:27.964504 23998 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:27.968261 23998 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 1.
I20260806 07:48:27.968806 23828 leader_election.cc:304] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328; no voters:
I20260806 07:48:27.969220 24030 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:48:27.976074 24030 raft_consensus.cc:697] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 LEADER]: Becoming Leader. State: Replica: c91b50fc26f949c4bcbf1c708b3e4328, State: Running, Role: LEADER
I20260806 07:48:27.976727 24030 consensus_queue.cc:237] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:27.981653 24056 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91b50fc26f949c4bcbf1c708b3e4328. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:27.981990 24056 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:27.993942 24058 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:27.996419 24058 catalog_manager.cc:1520] Initializing Kudu cluster ID...
W20260806 07:48:28.011230 23976 tablet.cc:2406] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:48:28.011513 23998 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Refusing update from remote peer c91b50fc26f949c4bcbf1c708b3e4328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:48:28.013084 24056 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:48:28.034162 24035 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91b50fc26f949c4bcbf1c708b3e4328. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.035292 24059 mvcc.cc:204] Tried to move back new op lower bound from 7315466272803840000 to 7315466272678342656. Current Snapshot: MvccSnapshot[applied={T|T < 7315466272803840000}]
I20260806 07:48:28.034463 24035 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:28.037129 24060 mvcc.cc:204] Tried to move back new op lower bound from 7315466272803840000 to 7315466272678342656. Current Snapshot: MvccSnapshot[applied={T|T < 7315466272803840000}]
I20260806 07:48:28.038703 24035 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.038900 24035 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:28.038964 24056 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.039522 24056 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:28.040477 23951 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } attempt: 5
I20260806 07:48:28.047091 24058 catalog_manager.cc:1383] Generated new cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:28.047199 24058 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:28.063127 23951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Bootstrap starting.
I20260806 07:48:28.067646 23951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:28.069253 23951 log.cc:826] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:28.088779 23951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: No bootstrap required, opened a new log
I20260806 07:48:28.110891 23951 raft_consensus.cc:359] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:28.111915 23951 raft_consensus.cc:385] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:28.114221 23951 raft_consensus.cc:740] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45e2ed1d0844fe1a0de0953d5a0340b, State: Initialized, Role: FOLLOWER
I20260806 07:48:28.116935 24058 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:48:28.116927 23951 consensus_queue.cc:260] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:28.118695 24058 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:28.135676 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" is_pre_election: true
I20260806 07:48:28.136257 23926 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 0.
I20260806 07:48:28.136615 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
I20260806 07:48:28.136847 23925 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:28.137903 24058 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Generated new TSK 0
I20260806 07:48:28.139093 24058 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:28.141506 23925 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 1.
I20260806 07:48:28.142341 23924 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 1 FOLLOWER]: Refusing update from remote peer c91b50fc26f949c4bcbf1c708b3e4328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:48:28.144012 23951 sys_catalog.cc:565] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:28.144456 24030 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:48:28.145989 24065 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.146306 24065 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:28.162302 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:28.197984 24065 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91b50fc26f949c4bcbf1c708b3e4328. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.198287 24065 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:28.198110 24071 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.198665 24071 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:28.295161 24082 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 07:48:28.542938 24030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:28.543382 24030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: LEADER
W20260806 07:48:28.621685 24068 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:28.622020 24068 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:28.622175 24068 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:28.630542 24068 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:28.630735 24068 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:28.640836 24068 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:28.642580 24068 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:28.644635 24068 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:28.654819 24088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:28.654838 24089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:28.655810 24091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:28.656225 24068 server_base.cc:1061] running on GCE node
I20260806 07:48:28.656886 24068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:28.657918 24068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:28.659147 24068 hybrid_clock.cc:648] HybridClock initialized: now 1786002508659134 us; error 59 us; skew 500 ppm
I20260806 07:48:28.659478 24068 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:28.661789 24068 webserver.cc:533] Webserver started at http://127.20.202.193:45859/ using document root <none> and password file <none>
I20260806 07:48:28.662595 24068 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:28.662724 24068 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:28.663128 24068 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:28.665681 24068 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:28.666500 24068 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:28.671954 24068 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 07:48:28.675487 24097 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:28.676857 24068 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:28.677038 24068 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:28.677286 24068 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:28.697455 24068 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:28.698422 24068 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:28.698719 24068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:28.699808 24068 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:28.701716 24068 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:28.701834 24068 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:28.701967 24068 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:28.702101 24068 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:28.736755 24068 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:39905
I20260806 07:48:28.736842 24210 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:39905 every 8 connection(s)
I20260806 07:48:28.738327 24068 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:28.740995 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24068
I20260806 07:48:28.741211 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:48:28.748768 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:28.785193 24213 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:28.785307 24211 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:28.785662 24211 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:28.785671 24213 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:28.786875 24211 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:28.788056 24213 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:28.790129 23988 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:39905)
I20260806 07:48:28.790400 23916 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:39905)
I20260806 07:48:28.793218 24212 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:28.793385 24212 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:28.793895 24212 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:28.795364 23844 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:39905)
I20260806 07:48:28.797067 23844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:50553
I20260806 07:48:28.860579 24054 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:28.860705 24054 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: loading cluster ID for follower catalog manager: success
I20260806 07:48:28.864490 24054 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: acquiring CA information for follower catalog manager: success
I20260806 07:48:28.866943 24054 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:48:29.127491 24217 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:29.127795 24217 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:29.127959 24217 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:29.136358 24217 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:29.136540 24217 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:48:29.147032 24217 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:29.148772 24217 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:29.150753 24217 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:29.160925 24226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:29.161127 24229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:29.160854 24227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:29.162016 24217 server_base.cc:1061] running on GCE node
I20260806 07:48:29.162668 24217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:29.163769 24217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:29.164988 24217 hybrid_clock.cc:648] HybridClock initialized: now 1786002509164924 us; error 63 us; skew 500 ppm
I20260806 07:48:29.165341 24217 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:29.167613 24217 webserver.cc:533] Webserver started at http://127.20.202.194:32933/ using document root <none> and password file <none>
I20260806 07:48:29.168395 24217 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:29.168535 24217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:29.168916 24217 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:29.171433 24217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "a57bd35ad6e949e587ac4de612e0599d"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:29.172223 24217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "a57bd35ad6e949e587ac4de612e0599d"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:29.177891 24217 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:48:29.181406 24235 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:29.183061 24217 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:29.183246 24217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "a57bd35ad6e949e587ac4de612e0599d"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:29.183480 24217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:29.204864 24217 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:29.205832 24217 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:29.206117 24217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:29.207082 24217 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:29.208809 24217 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:29.208933 24217 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:29.209071 24217 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:29.209162 24217 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:29.244220 24217 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:42513
I20260806 07:48:29.244309 24348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:42513 every 8 connection(s)
I20260806 07:48:29.246012 24217 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:48:29.252543 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24217
I20260806 07:48:29.252861 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:48:29.260512 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:29.266089 24349 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:29.266644 24349 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:29.266928 24350 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:29.267096 24350 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:29.268069 24349 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:29.268096 24350 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:29.269961 23844 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:42513)
I20260806 07:48:29.270722 23988 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:42513)
I20260806 07:48:29.271096 23844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:52945
I20260806 07:48:29.272109 24351 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:29.272239 24351 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:29.272780 24351 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:29.273978 23916 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:42513)
I20260806 07:48:29.300123 24082 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:29.300261 24082 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: loading cluster ID for follower catalog manager: success
I20260806 07:48:29.304225 24082 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: acquiring CA information for follower catalog manager: success
I20260806 07:48:29.306705 24082 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:48:29.613864 24356 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:29.614204 24356 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:29.614339 24356 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:29.622686 24356 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:29.622889 24356 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:48:29.632668 24356 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:29.634407 24356 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:29.636286 24356 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:29.646029 24362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:29.647629 24363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:29.648373 24365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:29.648910 24356 server_base.cc:1061] running on GCE node
I20260806 07:48:29.649770 24356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:29.650980 24356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:29.652227 24356 hybrid_clock.cc:648] HybridClock initialized: now 1786002509652143 us; error 71 us; skew 500 ppm
I20260806 07:48:29.652587 24356 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:29.655488 24356 webserver.cc:533] Webserver started at http://127.20.202.195:42137/ using document root <none> and password file <none>
I20260806 07:48:29.656527 24356 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:29.656663 24356 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:29.657088 24356 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:29.660315 24356 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "cb67fa1840f04f67a8838927e5006d43"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:29.661314 24356 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "cb67fa1840f04f67a8838927e5006d43"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:29.668388 24356 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 07:48:29.672947 24371 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:29.674672 24356 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:29.674945 24356 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "cb67fa1840f04f67a8838927e5006d43"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:29.675211 24356 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:29.703927 24356 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:29.704957 24356 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:29.705225 24356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:29.706300 24356 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:29.707969 24356 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:29.708055 24356 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:29.708138 24356 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:29.708194 24356 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:29.743458 24356 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:38175
I20260806 07:48:29.743544 24484 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:38175 every 8 connection(s)
I20260806 07:48:29.745203 24356 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:48:29.747259 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24356
I20260806 07:48:29.747440 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:48:29.764214 24486 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:29.764309 24485 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:29.764789 24486 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:29.764789 24485 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:29.765933 24487 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:29.766108 24487 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:29.766341 24485 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:29.766590 24487 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:29.766939 24486 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:29.768395 23916 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:38175)
I20260806 07:48:29.768393 23844 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:38175)
I20260806 07:48:29.769131 23988 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:38175)
I20260806 07:48:29.769443 23844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:51797
I20260806 07:48:29.774649 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:29.782013 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:29.782228 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23813
I20260806 07:48:29.783195 23874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:29.799821 24212 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
I20260806 07:48:29.893970 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23813
I20260806 07:48:29.917762 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--webserver_interface=127.20.202.254
--webserver_port=34511
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
W20260806 07:48:30.266001 24493 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:30.266342 24493 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:30.266431 24493 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:48:30.274259 24350 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
W20260806 07:48:30.275620 24493 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:30.275735 24493 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:30.275830 24493 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:30.275905 24493 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:48:30.276073 24350 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:39675 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:39675: connect: Connection refused (error 111)
I20260806 07:48:30.286819 24493 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=34511
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:30.288576 24493 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:30.290580 24493 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:30.299336 24500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:30.299697 24502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:30.299831 24499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:30.300791 24493 server_base.cc:1061] running on GCE node
I20260806 07:48:30.301581 24493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:30.303112 24493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:30.304315 24493 hybrid_clock.cc:648] HybridClock initialized: now 1786002510304277 us; error 34 us; skew 500 ppm
I20260806 07:48:30.304795 24493 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:30.307052 24493 webserver.cc:533] Webserver started at http://127.20.202.254:34511/ using document root <none> and password file <none>
I20260806 07:48:30.307850 24493 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:30.307956 24493 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:30.313496 24493 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.001s
I20260806 07:48:30.316442 24508 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:30.317958 24493 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:30.318130 24493 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
format_stamp: "Formatted at 2026-08-06 07:48:26 on dist-test-slave-g350"
I20260806 07:48:30.318866 24493 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:30.363722 24493 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:30.364676 24493 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:30.364988 24493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:30.386620 24493 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39675
I20260806 07:48:30.386653 24559 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39675 every 8 connection(s)
I20260806 07:48:30.388346 24493 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:30.393033 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24493
I20260806 07:48:30.393373 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24068
I20260806 07:48:30.396565 24560 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:30.400993 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap starting.
I20260806 07:48:30.403627 24206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:30.425462 24560 log.cc:826] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:30.436410 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:30.437063 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap complete.
I20260806 07:48:30.447544 24560 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:30.448549 24560 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91b50fc26f949c4bcbf1c708b3e4328, State: Initialized, Role: FOLLOWER
I20260806 07:48:30.449538 24560 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:30.450521 24566 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:30.450817 24566 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:30.451905 24560 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:30.473573 24577 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:30.473791 24577 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: loading cluster ID for follower catalog manager: success
I20260806 07:48:30.478470 24577 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: acquiring CA information for follower catalog manager: success
I20260806 07:48:30.482640 24577 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:30.515196 24493 master_runner.cc:264] No leader master found from master c91b50fc26f949c4bcbf1c708b3e4328
I20260806 07:48:30.515383 24493 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:30.524509 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24068
I20260806 07:48:30.543861 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:39905
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--webserver_port=45859
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:30.772367 24486 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
I20260806 07:48:30.784991 24525 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "cb67fa1840f04f67a8838927e5006d43" instance_seqno: 1786002509734595) as {username='slave'} at 127.20.202.195:40149; Asking this server to re-register.
I20260806 07:48:30.786166 24486 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:30.788056 24525 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:38175)
I20260806 07:48:30.870450 24585 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c91b50fc26f949c4bcbf1c708b3e4328)
I20260806 07:48:30.870750 24585 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:30.872637 24585 leader_election.cc:290] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675), a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855)
I20260806 07:48:30.874635 23924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" is_pre_election: true
I20260806 07:48:30.880932 24535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" is_pre_election: true
I20260806 07:48:30.881515 24535 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91518017b4e465fa782aa91e17b8de9 in term 1.
I20260806 07:48:30.882184 23971 leader_election.cc:304] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328; no voters: a45e2ed1d0844fe1a0de0953d5a0340b
I20260806 07:48:30.882691 24585 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:48:30.882835 24585 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c91b50fc26f949c4bcbf1c708b3e4328)
I20260806 07:48:30.882978 24585 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:30.885088 24585 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:30.885772 24585 leader_election.cc:290] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 2 election: Requested vote from peers c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675), a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855)
I20260806 07:48:30.886297 24535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
I20260806 07:48:30.886431 23924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
I20260806 07:48:30.886513 24535 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:30.890625 24535 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91518017b4e465fa782aa91e17b8de9 in term 2.
I20260806 07:48:30.891103 23971 leader_election.cc:304] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328; no voters: a45e2ed1d0844fe1a0de0953d5a0340b
I20260806 07:48:30.891448 24585 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:48:30.891898 24585 raft_consensus.cc:697] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 LEADER]: Becoming Leader. State: Replica: c91518017b4e465fa782aa91e17b8de9, State: Running, Role: LEADER
I20260806 07:48:30.892436 24585 consensus_queue.cc:237] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:30.894516 24587 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91518017b4e465fa782aa91e17b8de9. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:30.894865 24587 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:30.895641 24589 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:30.897794 24589 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:30.898615 24589 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:30.898689 24589 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:30.899828 24589 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:30.900647 24589 catalog_manager.cc:6081] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Loaded TSK: 0
I20260806 07:48:30.901628 24589 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260806 07:48:30.906347 24583 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:30.906693 24583 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:30.906831 24583 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:30.915395 24583 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:30.915663 24583 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:30.926044 24583 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:39905
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=45859
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:30.927678 24583 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:30.929554 24583 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:30.943871 24594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:30.943830 24595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:30.943894 24597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:30.944653 24583 server_base.cc:1061] running on GCE node
I20260806 07:48:30.945580 24583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:30.946648 24583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:30.947886 24583 hybrid_clock.cc:648] HybridClock initialized: now 1786002510947823 us; error 59 us; skew 500 ppm
I20260806 07:48:30.948222 24583 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:30.950737 24583 webserver.cc:533] Webserver started at http://127.20.202.193:45859/ using document root <none> and password file <none>
I20260806 07:48:30.951656 24583 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:30.951843 24583 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:30.957962 24583 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 07:48:30.960708 24603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:30.962194 24583 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:30.962378 24583 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:30.963141 24583 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:30.984340 24583 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:30.985268 24583 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:30.985579 24583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:30.986627 24583 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:30.988377 24583 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:30.988484 24583 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:30.988638 24583 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:30.988742 24583 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:31.024710 24583 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:39905
I20260806 07:48:31.024765 24716 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:39905 every 8 connection(s)
I20260806 07:48:31.026348 24583 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:31.028131 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24583
I20260806 07:48:31.028466 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24583
I20260806 07:48:31.057809 24712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:31.108088 24729 raft_consensus.cc:493] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c91b50fc26f949c4bcbf1c708b3e4328)
I20260806 07:48:31.108352 24729 raft_consensus.cc:515] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:31.109820 24729 leader_election.cc:290] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675), c91518017b4e465fa782aa91e17b8de9 (127.20.202.252:33895)
I20260806 07:48:31.111156 23998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c91518017b4e465fa782aa91e17b8de9" is_pre_election: true
I20260806 07:48:31.117470 24535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" is_pre_election: true
I20260806 07:48:31.117867 24535 raft_consensus.cc:2393] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a45e2ed1d0844fe1a0de0953d5a0340b in current term 2: Already voted for candidate c91518017b4e465fa782aa91e17b8de9 in this term.
I20260806 07:48:31.118579 23900 leader_election.cc:304] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45e2ed1d0844fe1a0de0953d5a0340b; no voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328
I20260806 07:48:31.119055 24729 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:31.121181 24729 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:48:31.147246 24717 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:31.147707 24717 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:31.148967 24717 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:31.149340 24718 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:31.149513 24718 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:31.149749 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24583
I20260806 07:48:31.169317 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:39905
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--webserver_port=45859
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
W20260806 07:48:31.169708 24726 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:48:31.042771 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:53323 (local address 127.20.202.253:45855)
0806 07:48:31.052548 (+ 9777us) server_negotiation.cc:207] Beginning negotiation
0806 07:48:31.052555 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:48:31.052575 (+ 20us) server_negotiation.cc:408] Connection header received
0806 07:48:31.052640 (+ 65us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:48:31.052646 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:48:31.052727 (+ 81us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:48:31.052958 (+ 231us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:48:31.054405 (+ 1447us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:48:31.055997 (+ 1592us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:48:31.058300 (+ 2303us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:48:31.059056 (+ 756us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:48:31.168747 (+109691us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:53323: BlockingRecv error: recv got EOF from 127.20.202.193:53323 (error 108)
Metrics: {"server-negotiator.queue_time_us":9623,"thread_start_us":9519,"threads_started":1}
I20260806 07:48:31.283995 24350 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:31.284236 24350 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
I20260806 07:48:31.285173 24525 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a57bd35ad6e949e587ac4de612e0599d" instance_seqno: 1786002509235918) as {username='slave'} at 127.20.202.194:47313; Asking this server to re-register.
I20260806 07:48:31.285772 24350 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:31.286872 24525 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:42513)
I20260806 07:48:31.302038 24535 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Refusing update from remote peer c91518017b4e465fa782aa91e17b8de9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:48:31.302995 24587 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:48:31.307471 24734 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91518017b4e465fa782aa91e17b8de9. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:31.307857 24734 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:31.309257 24585 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:31.309566 24585 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:31.309957 23924 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 2 FOLLOWER]: Refusing update from remote peer c91518017b4e465fa782aa91e17b8de9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:48:31.310621 24734 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:31.310832 24734 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:31.310762 24587 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:48:31.314364 24729 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91518017b4e465fa782aa91e17b8de9. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:31.314241 24740 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:31.314638 24729 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:31.314742 24740 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:48:31.547493 24732 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:31.547897 24732 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:31.548054 24732 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:48:31.548166 24732 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:31.557466 24732 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:31.557780 24732 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:31.568552 24732 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:39905
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=45859
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:31.570592 24732 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:31.572536 24732 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:31.583516 24746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:31.583568 24745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:31.585186 24748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:31.585583 24732 server_base.cc:1061] running on GCE node
I20260806 07:48:31.586212 24732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:31.587095 24732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:31.588339 24732 hybrid_clock.cc:648] HybridClock initialized: now 1786002511588280 us; error 40 us; skew 500 ppm
I20260806 07:48:31.588649 24732 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:31.590876 24732 webserver.cc:533] Webserver started at http://127.20.202.193:45859/ using document root <none> and password file <none>
I20260806 07:48:31.591612 24732 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:31.591771 24732 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:31.597168 24732 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 07:48:31.599941 24754 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:31.601402 24732 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:31.601544 24732 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:31.602444 24732 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:31.619110 24732 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:31.619953 24732 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:31.620203 24732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:31.621268 24732 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:31.623018 24732 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:31.623116 24732 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:31.623202 24732 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:31.623265 24732 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:31.623939 24732 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:48:31.624071 24732 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:48:31.624167 24732 fault_injection.cc:59] injecting fault for kudu (pid 24732): FLAGS_fault_before_start (process will exit)
I20260806 07:48:31.641678 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:39905
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--webserver_port=45859
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:31.752708 24587 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:31.753026 24587 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: LEADER
W20260806 07:48:32.002094 24762 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:32.002490 24762 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:32.002677 24762 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:32.017774 24762 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:32.018100 24762 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:32.031332 24762 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:39905
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=45859
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:32.033586 24762 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:32.035890 24762 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:32.046378 24771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:32.046336 24769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:32.046303 24768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:32.047705 24762 server_base.cc:1061] running on GCE node
I20260806 07:48:32.048381 24762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:32.049427 24762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:32.050632 24762 hybrid_clock.cc:648] HybridClock initialized: now 1786002512050607 us; error 75 us; skew 500 ppm
I20260806 07:48:32.050961 24762 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:32.053392 24762 webserver.cc:533] Webserver started at http://127.20.202.193:45859/ using document root <none> and password file <none>
I20260806 07:48:32.054287 24762 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:32.054394 24762 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:32.061200 24762 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 07:48:32.064390 24777 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:32.066110 24762 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:32.066313 24762 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:32.067134 24762 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:32.082238 24762 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:32.083105 24762 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:32.083415 24762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:32.084430 24762 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:32.086181 24762 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:32.086288 24762 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:32.086434 24762 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:32.086513 24762 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:32.122112 24762 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:39905
I20260806 07:48:32.122138 24890 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:39905 every 8 connection(s)
I20260806 07:48:32.123755 24762 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:32.124828 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24762
I20260806 07:48:32.125213 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24762
I20260806 07:48:32.158617 24892 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:32.158754 24891 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:32.159092 24892 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:32.159093 24891 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:32.160379 24892 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:32.160676 24891 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:32.161872 24886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:32.161922 24893 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:32.162060 24893 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:32.162564 24525 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:39905)
I20260806 07:48:32.163599 23988 ts_manager.cc:194] Re-registered known tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:39905)
I20260806 07:48:32.166045 23988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:46901
I20260806 07:48:32.252213 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24762
I20260806 07:48:32.271291 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24217
I20260806 07:48:32.277064 24349 heartbeater.cc:499] Master 127.20.202.252:33895 was elected leader, sending a full tablet report...
I20260806 07:48:32.280247 24344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:32.366676 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24217
I20260806 07:48:32.385417 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24356
I20260806 07:48:32.404106 24480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:32.490347 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24356
I20260806 07:48:32.509217 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24493
I20260806 07:48:32.515858 24555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:32.622560 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24493
I20260806 07:48:32.645767 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23883
I20260806 07:48:32.646948 23946 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:48:32.751863 23971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39675: connect: Connection refused (error 111)
I20260806 07:48:32.757994 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23883
W20260806 07:48:32.759025 23971 consensus_peers.cc:597] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 -> Peer c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675): Couldn't send request to peer c91b50fc26f949c4bcbf1c708b3e4328. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:48:32.780529 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23953
I20260806 07:48:32.781929 24018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:32.892064 23953 master_runner.cc:187] Error getting master registration for 127.20.202.254:39675: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39675: connect: Connection refused (error 111),
I20260806 07:48:32.892146 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23953
2026-08-06T07:48:32Z chronyd exiting
2026-08-06T07:48:32Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:32Z Disabled control of system clock
I20260806 07:48:32.958626 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
W20260806 07:48:33.299296 24917 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:33.299651 24917 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:33.299795 24917 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:33.308437 24917 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:33.308533 24917 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:33.308622 24917 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:33.308714 24917 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:33.318962 24917 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:33.320691 24917 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:33.322624 24917 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:33.331378 24923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:33.331346 24922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:33.332194 24925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:33.332674 24917 server_base.cc:1061] running on GCE node
I20260806 07:48:33.333447 24917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:33.334942 24917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:33.336187 24917 hybrid_clock.cc:648] HybridClock initialized: now 1786002513336151 us; error 63 us; skew 500 ppm
I20260806 07:48:33.336577 24917 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:33.338847 24917 webserver.cc:533] Webserver started at http://127.20.202.254:39099/ using document root <none> and password file <none>
I20260806 07:48:33.339629 24917 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:33.339761 24917 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:33.345484 24917 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:48:33.348510 24931 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:33.349954 24917 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:33.350167 24917 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
format_stamp: "Formatted at 2026-08-06 07:48:26 on dist-test-slave-g350"
I20260806 07:48:33.350896 24917 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:33.375371 24917 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:33.376374 24917 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:33.376705 24917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:33.400427 24917 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39675
I20260806 07:48:33.400466 24982 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39675 every 8 connection(s)
I20260806 07:48:33.402390 24917 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:33.409304 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24917
I20260806 07:48:33.409533 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:48:33.410086 24983 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:33.410712 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45855
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
I20260806 07:48:33.413622 24983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap starting.
I20260806 07:48:33.430811 24983 log.cc:826] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:33.440231 24983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:33.440820 24983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap complete.
I20260806 07:48:33.448084 24983 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:33.448830 24983 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91b50fc26f949c4bcbf1c708b3e4328, State: Initialized, Role: FOLLOWER
I20260806 07:48:33.449529 24983 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:33.450568 24987 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:33.450946 24987 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:33.451692 24983 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:33.462735 24998 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:33.462862 24998 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: loading cluster ID for follower catalog manager: success
I20260806 07:48:33.468075 24998 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: acquiring CA information for follower catalog manager: success
I20260806 07:48:33.473402 24998 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:33.485093 24917 master_runner.cc:187] Error getting master registration for 127.20.202.253:45855: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45855: connect: Connection refused (error 111),
I20260806 07:48:33.488816 24917 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:33.780481 24985 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:33.780808 24985 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:33.780959 24985 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:33.789724 24985 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:33.789824 24985 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:33.789917 24985 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:33.789989 24985 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:33.800705 24985 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:33.802634 24985 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:33.804584 24985 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:33.813344 25007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:33.813422 25006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:33.814777 25009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:33.815284 24985 server_base.cc:1061] running on GCE node
I20260806 07:48:33.816092 24985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:33.817482 24985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:33.818725 24985 hybrid_clock.cc:648] HybridClock initialized: now 1786002513818661 us; error 55 us; skew 500 ppm
I20260806 07:48:33.819123 24985 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:33.821473 24985 webserver.cc:533] Webserver started at http://127.20.202.253:34763/ using document root <none> and password file <none>
I20260806 07:48:33.822243 24985 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:33.822402 24985 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:33.827965 24985 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:48:33.830962 25015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:33.832448 24985 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:33.832615 24985 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:33.833300 24985 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:33.848477 24985 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:33.849334 24985 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:33.849710 24985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:33.870754 24985 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:45855
I20260806 07:48:33.870774 25066 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:45855 every 8 connection(s)
I20260806 07:48:33.872473 24985 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:48:33.872984 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24985
I20260806 07:48:33.873210 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:48:33.874416 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33895
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
I20260806 07:48:33.879822 25067 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:33.884100 25067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Bootstrap starting.
I20260806 07:48:33.907968 25067 log.cc:826] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:33.918897 25067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:33.919556 25067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: Bootstrap complete.
I20260806 07:48:33.929212 25067 raft_consensus.cc:359] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:33.930162 25067 raft_consensus.cc:740] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45e2ed1d0844fe1a0de0953d5a0340b, State: Initialized, Role: FOLLOWER
I20260806 07:48:33.930945 25067 consensus_queue.cc:260] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:33.931891 25071 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:33.932224 25071 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:33.933140 25067 sys_catalog.cc:565] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:33.949991 25082 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:33.950114 25082 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: loading cluster ID for follower catalog manager: success
I20260806 07:48:33.954255 25082 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: acquiring CA information for follower catalog manager: success
I20260806 07:48:33.957206 25082 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:33.973186 24985 master_runner.cc:187] Error getting master registration for 127.20.202.252:33895: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33895: connect: Connection refused (error 111),
I20260806 07:48:33.976953 24985 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:34.271404 25069 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:34.271802 25069 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:34.271938 25069 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:34.280578 25069 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:34.280715 25069 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:34.280838 25069 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:34.280921 25069 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:34.291468 25069 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:34.293185 25069 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:34.295356 25069 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:34.303963 25090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:34.304003 25091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:34.306164 25093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:34.306808 25069 server_base.cc:1061] running on GCE node
I20260806 07:48:34.307603 25069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:34.309113 25069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:34.310330 25069 hybrid_clock.cc:648] HybridClock initialized: now 1786002514310278 us; error 48 us; skew 500 ppm
I20260806 07:48:34.310685 25069 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:34.312956 25069 webserver.cc:533] Webserver started at http://127.20.202.252:46155/ using document root <none> and password file <none>
I20260806 07:48:34.313757 25069 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:34.313900 25069 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:34.319526 25069 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:48:34.322520 25099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:34.324273 25069 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:34.324525 25069 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "c91518017b4e465fa782aa91e17b8de9"
format_stamp: "Formatted at 2026-08-06 07:48:27 on dist-test-slave-g350"
I20260806 07:48:34.325273 25069 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:34.343546 25069 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:34.344478 25069 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:34.344781 25069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:34.367362 25069 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:33895
I20260806 07:48:34.367365 25150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:33895 every 8 connection(s)
I20260806 07:48:34.369084 25069 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:48:34.370061 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25069
I20260806 07:48:34.370270 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:48:34.378361 25151 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:34.382972 25151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Bootstrap starting.
I20260806 07:48:34.402402 25151 log.cc:826] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:34.411556 25151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:34.412271 25151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Bootstrap complete.
I20260806 07:48:34.418882 25151 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:34.419613 25151 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91518017b4e465fa782aa91e17b8de9, State: Initialized, Role: FOLLOWER
I20260806 07:48:34.420300 25151 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:34.421113 25156 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:34.421361 25156 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:34.422227 25151 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:34.433315 25167 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:34.433415 25167 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: loading cluster ID for follower catalog manager: success
I20260806 07:48:34.438149 25167 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: acquiring CA information for follower catalog manager: success
I20260806 07:48:34.440929 25167 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:34.458873 25069 master_runner.cc:264] No leader master found from master c91518017b4e465fa782aa91e17b8de9
I20260806 07:48:34.459077 25069 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:34.642124 25170 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:48:34.642447 25170 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:34.644447 25170 leader_election.cc:290] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855), c91518017b4e465fa782aa91e17b8de9 (127.20.202.252:33895)
I20260806 07:48:34.651743 25042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" is_pre_election: true
I20260806 07:48:34.652230 25042 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 2.
I20260806 07:48:34.652972 24932 leader_election.cc:304] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45e2ed1d0844fe1a0de0953d5a0340b, c91b50fc26f949c4bcbf1c708b3e4328; no voters:
I20260806 07:48:34.653741 25170 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:48:34.653925 25170 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:48:34.654029 25170 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:48:34.654842 25126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c91518017b4e465fa782aa91e17b8de9" is_pre_election: true
I20260806 07:48:34.655474 25126 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 2.
I20260806 07:48:34.658601 25170 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:34.659403 25170 leader_election.cc:290] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 3 election: Requested vote from peers a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855), c91518017b4e465fa782aa91e17b8de9 (127.20.202.252:33895)
I20260806 07:48:34.659863 25042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
I20260806 07:48:34.660082 25042 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:48:34.660239 25126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c91518017b4e465fa782aa91e17b8de9"
I20260806 07:48:34.660430 25126 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:48:34.665431 25042 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 3.
I20260806 07:48:34.665983 24932 leader_election.cc:304] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45e2ed1d0844fe1a0de0953d5a0340b, c91b50fc26f949c4bcbf1c708b3e4328; no voters:
I20260806 07:48:34.666148 25126 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91b50fc26f949c4bcbf1c708b3e4328 in term 3.
I20260806 07:48:34.666425 25170 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:48:34.666923 25170 raft_consensus.cc:697] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 LEADER]: Becoming Leader. State: Replica: c91b50fc26f949c4bcbf1c708b3e4328, State: Running, Role: LEADER
I20260806 07:48:34.667511 25170 consensus_queue.cc:237] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:34.670363 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91b50fc26f949c4bcbf1c708b3e4328. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:34.670708 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:34.671473 25175 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:34.673374 25175 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:34.674257 25175 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:34.674329 25175 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:34.675423 25175 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:34.676221 25175 catalog_manager.cc:6081] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Loaded TSK: 0
I20260806 07:48:34.677093 25175 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:34.688256 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:48:35.043572 25176 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:35.043951 25176 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:35.044095 25176 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:35.052886 25176 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:35.053143 25176 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:35.063912 25176 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:35.065678 25176 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:35.067587 25176 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:35.077704 25181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:35.077644 25182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:35.078356 25184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:35.079193 25176 server_base.cc:1061] running on GCE node
I20260806 07:48:35.079842 25176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:35.080837 25176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:35.082089 25176 hybrid_clock.cc:648] HybridClock initialized: now 1786002515082032 us; error 50 us; skew 500 ppm
I20260806 07:48:35.082432 25176 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:35.084666 25176 webserver.cc:533] Webserver started at http://127.20.202.193:36053/ using document root <none> and password file <none>
I20260806 07:48:35.085440 25176 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:35.085620 25176 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:35.091269 25176 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:48:35.093984 25190 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:35.095381 25176 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:35.095561 25176 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:35.096276 25176 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:35.108865 25176 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:35.109781 25176 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:35.110078 25176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:35.111111 25176 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:35.112958 25126 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Refusing update from remote peer c91b50fc26f949c4bcbf1c708b3e4328: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:48:35.113181 25176 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:35.113278 25176 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:35.113402 25176 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:35.113473 25176 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:35.114125 25173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:48:35.118172 25198 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91b50fc26f949c4bcbf1c708b3e4328. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:35.118511 25198 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:35.123162 25198 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:35.123345 25198 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:35.123522 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:35.123791 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:35.124672 25042 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 3 FOLLOWER]: Refusing update from remote peer c91b50fc26f949c4bcbf1c708b3e4328: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:48:35.125383 25173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:48:35.129671 25215 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:35.130016 25215 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:35.130486 25211 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91b50fc26f949c4bcbf1c708b3e4328. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:35.130760 25211 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:35.162696 25176 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41083
I20260806 07:48:35.162738 25312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41083 every 8 connection(s)
I20260806 07:48:35.164299 25176 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:35.171774 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25176
I20260806 07:48:35.172017 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:48:35.177227 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:35.182626 25314 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:35.182776 25313 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:35.182853 25315 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:35.183082 25313 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:35.183086 25314 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:35.183085 25315 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:35.184480 25313 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:35.184540 25315 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:35.184566 25314 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:35.187039 24948 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:35.187052 25116 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:35.188437 25032 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:35.188843 24948 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:47637
W20260806 07:48:35.524309 25323 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:35.524614 25323 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:35.524688 25323 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:35.534089 25323 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:35.534310 25323 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:48:35.545388 25323 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:35.547160 25323 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:35.549090 25323 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:35.559887 25329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:35.559861 25328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:35.560963 25331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:35.561070 25323 server_base.cc:1061] running on GCE node
I20260806 07:48:35.561723 25323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:35.562734 25323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:35.563985 25323 hybrid_clock.cc:648] HybridClock initialized: now 1786002515563897 us; error 72 us; skew 500 ppm
I20260806 07:48:35.564306 25323 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:35.566619 25323 webserver.cc:533] Webserver started at http://127.20.202.194:43757/ using document root <none> and password file <none>
I20260806 07:48:35.567363 25323 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:35.567478 25323 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:35.573009 25323 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 07:48:35.575518 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:35.575798 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:35.576701 25337 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:35.578325 25323 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:48:35.578470 25323 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "a57bd35ad6e949e587ac4de612e0599d"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:35.579177 25323 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:35.594038 25323 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:35.594808 25323 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:35.595059 25323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:35.595934 25323 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:35.597576 25323 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:35.597723 25323 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:48:35.597818 25323 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:35.597862 25323 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:35.634771 25323 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:36711
I20260806 07:48:35.634805 25450 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:36711 every 8 connection(s)
I20260806 07:48:35.636461 25323 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:48:35.645087 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25323
I20260806 07:48:35.645303 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:48:35.650638 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:35.654126 25453 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:35.654597 25453 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:35.655769 25453 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:35.654136 25451 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:35.656872 25451 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:35.657409 25451 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:35.658288 25032 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:36711)
I20260806 07:48:35.658741 25116 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:36711)
I20260806 07:48:35.661640 25452 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:35.661798 25452 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:35.662233 25452 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:35.663221 24948 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:36711)
I20260806 07:48:35.664356 24948 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:54205
W20260806 07:48:36.015957 25458 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:36.016281 25458 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:36.016363 25458 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:36.026172 25458 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:36.026412 25458 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:48:36.038429 25458 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:36.040241 25458 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:36.042300 25458 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:36.052569 25464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:36.052556 25463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:36.053306 25466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:36.054062 25458 server_base.cc:1061] running on GCE node
I20260806 07:48:36.054626 25458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:36.055685 25458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:36.056928 25458 hybrid_clock.cc:648] HybridClock initialized: now 1786002516056833 us; error 81 us; skew 500 ppm
I20260806 07:48:36.057276 25458 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:36.059547 25458 webserver.cc:533] Webserver started at http://127.20.202.195:43301/ using document root <none> and password file <none>
I20260806 07:48:36.060313 25458 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:36.060484 25458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:36.066195 25458 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:48:36.068892 25472 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:36.070499 25458 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:48:36.070698 25458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "cb67fa1840f04f67a8838927e5006d43"
format_stamp: "Formatted at 2026-08-06 07:48:29 on dist-test-slave-g350"
I20260806 07:48:36.071439 25458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:36.095755 25458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:36.096700 25458 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:36.097010 25458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:36.098021 25458 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:36.099823 25458 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:36.099942 25458 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:36.100095 25458 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:36.100167 25458 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:36.136242 25458 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:45527
I20260806 07:48:36.136294 25585 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:45527 every 8 connection(s)
I20260806 07:48:36.137876 25458 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:48:36.146333 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25458
I20260806 07:48:36.146552 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:48:36.155869 25587 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:36.156126 25588 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:36.156148 25586 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:36.156355 25588 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:36.156358 25587 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:36.156359 25586 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:36.157424 25586 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:36.157487 25587 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:36.157454 25588 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:36.159226 24948 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:45527)
I20260806 07:48:36.159219 25032 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:45527)
I20260806 07:48:36.159283 25116 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:45527)
I20260806 07:48:36.160169 24948 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:35833
I20260806 07:48:36.172178 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:36.180135 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:36.180361 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24917
I20260806 07:48:36.181322 24978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:36.191753 25314 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
I20260806 07:48:36.291597 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24917
I20260806 07:48:36.315910 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--webserver_interface=127.20.202.254
--webserver_port=39099
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895 with env {}
W20260806 07:48:36.658711 25594 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:36.659010 25594 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:36.659087 25594 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:48:36.666787 25452 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
W20260806 07:48:36.668448 25452 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:39675 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:39675: connect: Connection refused (error 111)
W20260806 07:48:36.668730 25594 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:36.668836 25594 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:36.668926 25594 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:36.669015 25594 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:36.679649 25594 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=39099
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:36.681375 25594 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:36.683269 25594 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:36.691455 25601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:36.691498 25600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:36.692271 25603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:36.692721 25594 server_base.cc:1061] running on GCE node
I20260806 07:48:36.693456 25594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:36.694846 25594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:36.696076 25594 hybrid_clock.cc:648] HybridClock initialized: now 1786002516696023 us; error 55 us; skew 500 ppm
I20260806 07:48:36.696430 25594 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:36.698642 25594 webserver.cc:533] Webserver started at http://127.20.202.254:39099/ using document root <none> and password file <none>
I20260806 07:48:36.699376 25594 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:36.699522 25594 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:36.705215 25594 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:36.708257 25609 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:36.709717 25594 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:36.709898 25594 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
format_stamp: "Formatted at 2026-08-06 07:48:26 on dist-test-slave-g350"
I20260806 07:48:36.710630 25594 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:36.752511 25594 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:36.753487 25594 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:36.753840 25594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:36.775403 25594 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39675
I20260806 07:48:36.775427 25660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39675 every 8 connection(s)
I20260806 07:48:36.777271 25594 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:36.784896 25661 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:36.786386 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25594
I20260806 07:48:36.786702 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25176
I20260806 07:48:36.788913 25661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap starting.
I20260806 07:48:36.796340 25308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:36.809106 25661 log.cc:826] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:36.818969 25661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:36.819550 25661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: Bootstrap complete.
I20260806 07:48:36.826959 25661 raft_consensus.cc:359] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:36.827747 25661 raft_consensus.cc:740] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91b50fc26f949c4bcbf1c708b3e4328, State: Initialized, Role: FOLLOWER
I20260806 07:48:36.828486 25661 consensus_queue.cc:260] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:36.829561 25667 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:36.829932 25667 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:36.830812 25661 sys_catalog.cc:565] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:36.841513 25678 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:36.841696 25678 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: loading cluster ID for follower catalog manager: success
I20260806 07:48:36.845983 25678 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: acquiring CA information for follower catalog manager: success
I20260806 07:48:36.848506 25678 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:36.895002 25594 master_runner.cc:264] No leader master found from master c91b50fc26f949c4bcbf1c708b3e4328
I20260806 07:48:36.895190 25594 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:36.902144 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25176
I20260806 07:48:36.927871 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41083
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--webserver_port=36053
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:37.162263 25587 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
I20260806 07:48:37.174226 25626 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "cb67fa1840f04f67a8838927e5006d43" instance_seqno: 1786002516127392) as {username='slave'} at 127.20.202.195:46179; Asking this server to re-register.
I20260806 07:48:37.175287 25587 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:37.177474 25626 ts_manager.cc:194] Registered new tserver with Master: cb67fa1840f04f67a8838927e5006d43 (127.20.202.195:45527)
W20260806 07:48:37.286877 25684 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:37.287205 25684 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:37.287286 25684 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:37.296411 25684 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:37.296587 25684 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:37.307850 25684 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41083
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=36053
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:37.309736 25684 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:37.311755 25684 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:37.322464 25690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:37.322674 25691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:37.322563 25693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:37.323770 25684 server_base.cc:1061] running on GCE node
I20260806 07:48:37.324435 25684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:37.325559 25684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:37.326898 25684 hybrid_clock.cc:648] HybridClock initialized: now 1786002517326784 us; error 48 us; skew 500 ppm
I20260806 07:48:37.327237 25684 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:37.329527 25684 webserver.cc:533] Webserver started at http://127.20.202.193:36053/ using document root <none> and password file <none>
I20260806 07:48:37.330399 25684 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:37.330521 25684 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:37.336983 25684 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 07:48:37.340160 25699 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:37.341696 25684 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:37.341892 25684 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:37.342661 25684 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:37.372305 25684 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:37.373224 25684 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:37.373535 25684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:37.374579 25684 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:37.376367 25684 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:37.376464 25684 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:37.376609 25684 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:37.376670 25684 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:37.412361 25684 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41083
I20260806 07:48:37.412454 25812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41083 every 8 connection(s)
I20260806 07:48:37.414022 25684 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:37.421026 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25684
I20260806 07:48:37.421417 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25684
I20260806 07:48:37.437068 25815 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:37.437453 25815 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:37.437935 25813 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:37.438052 25814 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:37.438105 25813 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:37.438261 25814 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:37.438880 25815 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:37.438901 25814 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:37.438953 25813 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:37.439683 25808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:37.528079 25825 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c91b50fc26f949c4bcbf1c708b3e4328)
I20260806 07:48:37.528419 25825 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:37.528656 25626 ts_manager.cc:194] Registered new tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:37.530109 25116 ts_manager.cc:194] Re-registered known tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:37.530094 25032 ts_manager.cc:194] Re-registered known tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:37.530593 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25684
I20260806 07:48:37.531093 25825 leader_election.cc:290] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675), a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855)
I20260806 07:48:37.533416 25042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" is_pre_election: true
I20260806 07:48:37.539304 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" is_pre_election: true
I20260806 07:48:37.539896 25636 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91518017b4e465fa782aa91e17b8de9 in term 3.
I20260806 07:48:37.540493 25100 leader_election.cc:304] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328; no voters: a45e2ed1d0844fe1a0de0953d5a0340b
I20260806 07:48:37.540951 25825 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 07:48:37.541091 25825 raft_consensus.cc:493] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c91b50fc26f949c4bcbf1c708b3e4328)
I20260806 07:48:37.541201 25825 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:48:37.544934 25825 raft_consensus.cc:515] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:37.545902 25825 leader_election.cc:290] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 4 election: Requested vote from peers c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675), a45e2ed1d0844fe1a0de0953d5a0340b (127.20.202.253:45855)
I20260806 07:48:37.546517 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c91b50fc26f949c4bcbf1c708b3e4328"
I20260806 07:48:37.546693 25042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c91518017b4e465fa782aa91e17b8de9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b"
I20260806 07:48:37.546783 25636 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:48:37.551182 25636 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91518017b4e465fa782aa91e17b8de9 in term 4.
I20260806 07:48:37.551699 25100 leader_election.cc:304] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328; no voters: a45e2ed1d0844fe1a0de0953d5a0340b
I20260806 07:48:37.552150 25825 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 4 FOLLOWER]: Leader election won for term 4
W20260806 07:48:37.552402 25016 connection.cc:573] server connection from 127.20.202.193:57173 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:37.552757 25825 raft_consensus.cc:697] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [term 4 LEADER]: Becoming Leader. State: Replica: c91518017b4e465fa782aa91e17b8de9, State: Running, Role: LEADER
I20260806 07:48:37.553331 25825 consensus_queue.cc:237] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:37.553653 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41083
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--webserver_port=36053
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260806 07:48:37.555380 25827 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91518017b4e465fa782aa91e17b8de9. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:37.555757 25827 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:37.556480 25829 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:37.558527 25829 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:37.559332 25829 catalog_manager.cc:1295] Loaded cluster ID: 536fb4e3d3b04f51bbbbd237d221701b
I20260806 07:48:37.559435 25829 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:37.560581 25829 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:37.561674 25829 catalog_manager.cc:6081] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9: Loaded TSK: 0
I20260806 07:48:37.562860 25829 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:37.647181 25831 raft_consensus.cc:493] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c91b50fc26f949c4bcbf1c708b3e4328)
I20260806 07:48:37.647446 25831 raft_consensus.cc:515] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } }
I20260806 07:48:37.649487 25831 leader_election.cc:290] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c91b50fc26f949c4bcbf1c708b3e4328 (127.20.202.254:39675), c91518017b4e465fa782aa91e17b8de9 (127.20.202.252:33895)
I20260806 07:48:37.658030 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" is_pre_election: true
I20260806 07:48:37.658473 25126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c91518017b4e465fa782aa91e17b8de9" is_pre_election: true
I20260806 07:48:37.658560 25636 raft_consensus.cc:2393] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a45e2ed1d0844fe1a0de0953d5a0340b in current term 4: Already voted for candidate c91518017b4e465fa782aa91e17b8de9 in this term.
I20260806 07:48:37.659353 25016 leader_election.cc:304] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45e2ed1d0844fe1a0de0953d5a0340b; no voters: c91518017b4e465fa782aa91e17b8de9, c91b50fc26f949c4bcbf1c708b3e4328
I20260806 07:48:37.659798 25831 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:48:37.662159 25831 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 07:48:37.675729 25452 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:37.676000 25452 heartbeater.cc:499] Master 127.20.202.254:39675 was elected leader, sending a full tablet report...
I20260806 07:48:37.677146 25626 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a57bd35ad6e949e587ac4de612e0599d" instance_seqno: 1786002515626029) as {username='slave'} at 127.20.202.194:42157; Asking this server to re-register.
I20260806 07:48:37.677872 25452 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:37.679140 25626 ts_manager.cc:194] Registered new tserver with Master: a57bd35ad6e949e587ac4de612e0599d (127.20.202.194:36711)
W20260806 07:48:37.909442 25830 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:37.909861 25830 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:37.910012 25830 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:48:37.910111 25830 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:37.918712 25830 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:37.918897 25830 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:37.929039 25830 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41083
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=36053
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:37.930840 25830 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:37.932874 25830 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:37.942972 25839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:37.946457 25840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:37.949607 25842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:37.950320 25830 server_base.cc:1061] running on GCE node
I20260806 07:48:37.950989 25830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:37.951890 25830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:37.953136 25830 hybrid_clock.cc:648] HybridClock initialized: now 1786002517953066 us; error 38 us; skew 500 ppm
I20260806 07:48:37.953501 25830 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:37.956749 25830 webserver.cc:533] Webserver started at http://127.20.202.193:36053/ using document root <none> and password file <none>
I20260806 07:48:37.957651 25830 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:37.957809 25830 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:37.963656 25830 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 07:48:37.966602 25848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:37.967974 25830 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:48:37.968150 25830 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:37.968916 25830 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:37.996451 25830 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:37.997412 25830 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:37.997745 25830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:37.998845 25830 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:38.000607 25830 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:38.000696 25830 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:38.000855 25830 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:38.000933 25830 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:38.001333 25830 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:48:38.001472 25830 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:48:38.001552 25830 fault_injection.cc:59] injecting fault for kudu (pid 25830): FLAGS_fault_before_start (process will exit)
I20260806 07:48:38.017946 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41083
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--webserver_port=36053
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:38.128660 25636 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [term 4 FOLLOWER]: Refusing update from remote peer c91518017b4e465fa782aa91e17b8de9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:48:38.129571 25857 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:48:38.133569 25858 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91518017b4e465fa782aa91e17b8de9. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:38.133967 25858 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:38.136093 25860 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:38.136240 25042 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [term 4 FOLLOWER]: Refusing update from remote peer c91518017b4e465fa782aa91e17b8de9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:48:38.136317 25860 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:38.142822 25858 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:38.143054 25858 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91b50fc26f949c4bcbf1c708b3e4328 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:38.151607 25862 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:48:38.154590 25868 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:38.154884 25868 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:38.154847 25831 sys_catalog.cc:455] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c91518017b4e465fa782aa91e17b8de9. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:38.155128 25831 sys_catalog.cc:458] T 00000000000000000000000000000000 P a45e2ed1d0844fe1a0de0953d5a0340b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:48:38.372675 25856 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:38.372973 25856 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:38.373139 25856 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:38.381682 25856 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:38.381897 25856 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:38.391979 25856 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41083
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=36053
--tserver_master_addrs=127.20.202.254:39675,127.20.202.253:45855,127.20.202.252:33895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:38.393807 25856 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:38.395757 25856 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:38.405490 25873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:38.405520 25874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:38.406666 25876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:38.406695 25856 server_base.cc:1061] running on GCE node
I20260806 07:48:38.407315 25856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:38.408226 25856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:38.409427 25856 hybrid_clock.cc:648] HybridClock initialized: now 1786002518409383 us; error 31 us; skew 500 ppm
I20260806 07:48:38.409773 25856 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:38.411875 25856 webserver.cc:533] Webserver started at http://127.20.202.193:36053/ using document root <none> and password file <none>
I20260806 07:48:38.412566 25856 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:38.412667 25856 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:38.418116 25856 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:48:38.420634 25882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:38.422139 25856 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:38.422299 25856 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7c26b77b4af343bd87445682f48711d5"
format_stamp: "Formatted at 2026-08-06 07:48:28 on dist-test-slave-g350"
I20260806 07:48:38.423048 25856 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:38.449558 25856 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:38.450404 25856 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:38.450665 25856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:38.451567 25856 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:38.453140 25856 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:38.453212 25856 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:38.453294 25856 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:38.453338 25856 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:38.488297 25856 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41083
I20260806 07:48:38.488343 25995 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41083 every 8 connection(s)
I20260806 07:48:38.489892 25856 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:38.499773 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25856
I20260806 07:48:38.500114 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25856
I20260806 07:48:38.516527 25998 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45855
I20260806 07:48:38.517081 25998 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:38.517294 25997 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39675
I20260806 07:48:38.517464 25997 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:38.518536 25998 heartbeater.cc:507] Master 127.20.202.253:45855 requested a full tablet report, sending...
I20260806 07:48:38.518558 25997 heartbeater.cc:507] Master 127.20.202.254:39675 requested a full tablet report, sending...
I20260806 07:48:38.520609 25626 ts_manager.cc:194] Re-registered known tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:38.522142 25032 ts_manager.cc:194] Re-registered known tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:38.523774 25991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:38.591390 25857 sys_catalog.cc:455] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "c91518017b4e465fa782aa91e17b8de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c91b50fc26f949c4bcbf1c708b3e4328" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39675 } } peers { permanent_uuid: "a45e2ed1d0844fe1a0de0953d5a0340b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45855 } } peers { permanent_uuid: "c91518017b4e465fa782aa91e17b8de9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33895 } } }
I20260806 07:48:38.591733 25857 sys_catalog.cc:458] T 00000000000000000000000000000000 P c91518017b4e465fa782aa91e17b8de9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:38.609503 25996 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33895
I20260806 07:48:38.609764 25996 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:38.610335 25996 heartbeater.cc:507] Master 127.20.202.252:33895 requested a full tablet report, sending...
I20260806 07:48:38.610932 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25856
I20260806 07:48:38.612277 25116 ts_manager.cc:194] Re-registered known tserver with Master: 7c26b77b4af343bd87445682f48711d5 (127.20.202.193:41083)
I20260806 07:48:38.614321 25116 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:41679
W20260806 07:48:38.630823 25100 connection.cc:573] server connection from 127.20.202.193:41679 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:38.631296 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25323
I20260806 07:48:38.640226 25446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:38.726517 25451 heartbeater.cc:499] Master 127.20.202.252:33895 was elected leader, sending a full tablet report...
I20260806 07:48:38.727900 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25323
W20260806 07:48:38.746673 25019 connection.cc:573] server connection from 127.20.202.194:57407 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:38.746739 25612 connection.cc:573] server connection from 127.20.202.194:42157 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:38.747001 25100 connection.cc:573] server connection from 127.20.202.194:33075 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:38.747280 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25458
I20260806 07:48:38.755725 25581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:38.850361 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25458
I20260806 07:48:38.869450 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25594
I20260806 07:48:38.876935 25656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:38.984572 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25594
W20260806 07:48:39.006763 25016 connection.cc:573] client connection to 127.20.202.254:39675 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:39.007072 24985 master_runner.cc:217] Error getting master consensus for 127.20.202.254:39675: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:39.007341 24985 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:39.007324 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24985
I20260806 07:48:39.008575 25062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:39.122197 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24985
I20260806 07:48:39.145219 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25069
I20260806 07:48:39.146479 25146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:39.254724 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25069
2026-08-06T07:48:39Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/1 (12891 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/2
2026-08-06T07:48:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:39Z Disabled control of system clock
2026-08-06T07:48:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:39Z Disabled control of system clock
2026-08-06T07:48:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:39Z Disabled control of system clock
2026-08-06T07:48:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:39Z Disabled control of system clock
2026-08-06T07:48:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:39Z Disabled control of system clock
I20260806 07:48:39.414584 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
W20260806 07:48:39.753043 26028 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:39.753407 26028 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:39.753558 26028 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:39.762063 26028 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:39.762168 26028 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:39.762291 26028 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:39.762367 26028 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:39.773430 26028 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:39.775190 26028 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:39.777069 26028 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:39.785177 26033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:39.785169 26034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:39.786096 26036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:39.786966 26028 server_base.cc:1061] running on GCE node
I20260806 07:48:39.787711 26028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:39.789284 26028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:39.792622 26028 hybrid_clock.cc:648] HybridClock initialized: now 1786002519792387 us; error 227 us; skew 500 ppm
I20260806 07:48:39.793016 26028 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:39.795186 26028 webserver.cc:533] Webserver started at http://127.20.202.254:35491/ using document root <none> and password file <none>
I20260806 07:48:39.796025 26028 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:39.796164 26028 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:39.796525 26028 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:39.799036 26028 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "39521fcda47b40639ccce746141fd04c"
format_stamp: "Formatted at 2026-08-06 07:48:39 on dist-test-slave-g350"
I20260806 07:48:39.799840 26028 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "39521fcda47b40639ccce746141fd04c"
format_stamp: "Formatted at 2026-08-06 07:48:39 on dist-test-slave-g350"
I20260806 07:48:39.805367 26028 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:48:39.809031 26042 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:39.810524 26028 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:39.810712 26028 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "39521fcda47b40639ccce746141fd04c"
format_stamp: "Formatted at 2026-08-06 07:48:39 on dist-test-slave-g350"
I20260806 07:48:39.810945 26028 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:39.837644 26028 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:39.838581 26028 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:39.838884 26028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:39.860011 26028 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39423
I20260806 07:48:39.860023 26093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39423 every 8 connection(s)
I20260806 07:48:39.861644 26028 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:39.863761 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26028
I20260806 07:48:39.864013 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:48:39.865362 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39911
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
I20260806 07:48:39.867049 26094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:39.874818 26094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:39.893294 26094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:39.894618 26046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111)
W20260806 07:48:39.896550 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39911: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111)
I20260806 07:48:39.945911 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } attempt: 1
W20260806 07:48:39.948068 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39911: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111)
I20260806 07:48:40.016387 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } attempt: 2
W20260806 07:48:40.018873 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39911: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111)
I20260806 07:48:40.110196 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } attempt: 3
W20260806 07:48:40.112452 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39911: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111)
W20260806 07:48:40.216281 26096 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:40.216601 26096 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:40.216682 26096 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:40.224942 26096 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:40.225036 26096 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:40.225095 26096 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:40.225140 26096 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:40.235180 26096 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:40.236743 26096 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:40.238641 26096 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:40.247160 26105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:40.247195 26104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:40.248051 26107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:40.248471 26096 server_base.cc:1061] running on GCE node
I20260806 07:48:40.249212 26096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:40.250890 26096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:40.252153 26096 hybrid_clock.cc:648] HybridClock initialized: now 1786002520252023 us; error 96 us; skew 500 ppm
I20260806 07:48:40.252552 26096 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:40.254819 26096 webserver.cc:533] Webserver started at http://127.20.202.253:43193/ using document root <none> and password file <none>
I20260806 07:48:40.255590 26096 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:40.255733 26096 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:40.255779 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } attempt: 4
I20260806 07:48:40.256175 26096 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
W20260806 07:48:40.257920 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39911: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111)
I20260806 07:48:40.258876 26096 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "956dbdc89b3b4221a4b2d95f757a405f"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:40.259619 26096 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "956dbdc89b3b4221a4b2d95f757a405f"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:40.264931 26096 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:48:40.268384 26113 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:40.269862 26096 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:40.270071 26096 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "956dbdc89b3b4221a4b2d95f757a405f"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:40.270288 26096 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:40.286434 26096 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:40.287322 26096 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:40.287611 26096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:40.308881 26096 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:39911
I20260806 07:48:40.308890 26165 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:39911 every 8 connection(s)
I20260806 07:48:40.310639 26096 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:48:40.313288 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26096
I20260806 07:48:40.313503 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:48:40.314616 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36323
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
I20260806 07:48:40.315133 26166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:40.321048 26166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:40.336901 26166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:40.345875 26166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:40.346972 26117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
W20260806 07:48:40.348495 26166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
I20260806 07:48:40.397801 26166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 1
W20260806 07:48:40.399920 26166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
I20260806 07:48:40.468223 26166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 2
W20260806 07:48:40.470692 26166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
I20260806 07:48:40.557222 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } attempt: 5
I20260806 07:48:40.561952 26166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 3
W20260806 07:48:40.564392 26166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
I20260806 07:48:40.564612 26094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:48:40.566640 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
I20260806 07:48:40.617882 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 1
W20260806 07:48:40.620026 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
W20260806 07:48:40.660946 26168 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:40.661288 26168 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:40.661428 26168 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:40.669847 26168 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:40.669930 26168 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:40.669986 26168 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:40.670110 26168 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:40.680627 26168 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:40.682335 26168 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:40.684204 26168 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:48:40.688277 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 2
W20260806 07:48:40.690878 26094 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
W20260806 07:48:40.693301 26176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:40.693332 26177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:40.693526 26179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:40.694440 26168 server_base.cc:1061] running on GCE node
I20260806 07:48:40.695184 26168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:40.696866 26168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:40.698195 26168 hybrid_clock.cc:648] HybridClock initialized: now 1786002520698050 us; error 110 us; skew 500 ppm
I20260806 07:48:40.698560 26168 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:40.700829 26168 webserver.cc:533] Webserver started at http://127.20.202.252:42307/ using document root <none> and password file <none>
I20260806 07:48:40.701583 26168 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:40.701772 26168 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:40.702149 26168 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:40.704530 26168 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "5ec1137a1dc7429ab1e4378a48a40cdf"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:40.705307 26168 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "5ec1137a1dc7429ab1e4378a48a40cdf"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:40.707624 26166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 4
W20260806 07:48:40.709754 26166 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111)
I20260806 07:48:40.711500 26168 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 07:48:40.714926 26185 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:40.716372 26168 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:48:40.716547 26168 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "5ec1137a1dc7429ab1e4378a48a40cdf"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:40.716820 26168 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:40.736534 26168 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:40.737493 26168 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:40.737810 26168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:40.758973 26236 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:36323 every 8 connection(s)
I20260806 07:48:40.758983 26168 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:36323
I20260806 07:48:40.760617 26168 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:48:40.765072 26237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:48:40.767808 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26168
I20260806 07:48:40.768067 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:48:40.772186 26237 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:40.787309 26237 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:40.793646 26237 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:48:40.797127 26094 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 3
I20260806 07:48:40.805925 26237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Bootstrap starting.
I20260806 07:48:40.808561 26094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap starting.
I20260806 07:48:40.809168 26237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:40.810616 26237 log.cc:826] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:40.812016 26094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:40.813023 26094 log.cc:826] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:40.813648 26237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: No bootstrap required, opened a new log
I20260806 07:48:40.815416 26094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: No bootstrap required, opened a new log
I20260806 07:48:40.819002 26237 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.819375 26237 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:40.819558 26237 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec1137a1dc7429ab1e4378a48a40cdf, State: Initialized, Role: FOLLOWER
I20260806 07:48:40.820221 26237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.821209 26246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:40.821646 26246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:40.821516 26094 raft_consensus.cc:359] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.821910 26246 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:48:40.822033 26094 raft_consensus.cc:385] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:40.822178 26246 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.822297 26094 raft_consensus.cc:740] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39521fcda47b40639ccce746141fd04c, State: Initialized, Role: FOLLOWER
I20260806 07:48:40.822605 26237 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:40.823359 26094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.823890 26246 leader_election.cc:290] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39521fcda47b40639ccce746141fd04c (127.20.202.254:39423), 956dbdc89b3b4221a4b2d95f757a405f (127.20.202.253:39911)
I20260806 07:48:40.824828 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:40.825668 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:40.826208 26094 sys_catalog.cc:565] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:40.827891 26069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39521fcda47b40639ccce746141fd04c" is_pre_election: true
I20260806 07:48:40.828507 26069 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec1137a1dc7429ab1e4378a48a40cdf in term 0.
I20260806 07:48:40.829888 26189 leader_election.cc:304] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39521fcda47b40639ccce746141fd04c, 5ec1137a1dc7429ab1e4378a48a40cdf; no voters:
I20260806 07:48:40.830446 26247 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:48:40.830592 26247 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:48:40.830758 26247 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:40.835558 26247 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.836716 26247 leader_election.cc:290] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [CANDIDATE]: Term 1 election: Requested vote from peers 39521fcda47b40639ccce746141fd04c (127.20.202.254:39423), 956dbdc89b3b4221a4b2d95f757a405f (127.20.202.253:39911)
I20260806 07:48:40.837347 26069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39521fcda47b40639ccce746141fd04c"
I20260806 07:48:40.837899 26069 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:48:40.841722 26048 tablet.cc:2406] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:48:40.843297 26069 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec1137a1dc7429ab1e4378a48a40cdf in term 1.
W20260806 07:48:40.846336 26270 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:40.846469 26270 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:48:40.847383 26189 leader_election.cc:304] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39521fcda47b40639ccce746141fd04c, 5ec1137a1dc7429ab1e4378a48a40cdf; no voters:
I20260806 07:48:40.847704 26247 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:48:40.847908 26247 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 LEADER]: Becoming Leader. State: Replica: 5ec1137a1dc7429ab1e4378a48a40cdf, State: Running, Role: LEADER
I20260806 07:48:40.848416 26247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:40.851006 26168 master_runner.cc:187] Error getting master registration for 127.20.202.253:39911: OK, instance_id { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" instance_seqno: 1786002520303078 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:48:40.864791 26168 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:40.867370 26246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ec1137a1dc7429ab1e4378a48a40cdf. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:40.867650 26246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:40.868527 26274 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:40.871042 26274 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:40.855621 26028 master_runner.cc:187] Error getting master registration for 127.20.202.253:39911: OK, instance_id { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" instance_seqno: 1786002520303078 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:48:40.875775 26028 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:40.886646 26265 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:48:40.886817 26265 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:48:40.890529 26069 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Refusing update from remote peer 5ec1137a1dc7429ab1e4378a48a40cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:48:40.891603 26246 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:48:40.909400 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ec1137a1dc7429ab1e4378a48a40cdf. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:40.909752 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:40.914171 26275 mvcc.cc:204] Tried to move back new op lower bound from 7315466325553815552 to 7315466325399764992. Current Snapshot: MvccSnapshot[applied={T|T < 7315466325553815552}]
I20260806 07:48:40.915820 26246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:40.916075 26246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:40.919688 26274 catalog_manager.cc:1383] Generated new cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:40.919816 26274 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:40.930562 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:40.930851 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:40.961714 26274 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:48:40.965421 26274 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:40.986896 26274 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Generated new TSK 0
I20260806 07:48:40.988114 26274 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:41.000996 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:41.008975 26166 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } attempt: 5
I20260806 07:48:41.026556 26166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Bootstrap starting.
I20260806 07:48:41.036130 26166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Neither blocks nor log segments found. Creating new log.
I20260806 07:48:41.039184 26166 log.cc:826] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:41.054183 26166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: No bootstrap required, opened a new log
I20260806 07:48:41.072095 26166 raft_consensus.cc:359] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:41.073786 26166 raft_consensus.cc:385] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:48:41.074460 26166 raft_consensus.cc:740] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956dbdc89b3b4221a4b2d95f757a405f, State: Initialized, Role: FOLLOWER
I20260806 07:48:41.078013 26166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:41.092289 26282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:41.092707 26282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:41.095911 26141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956dbdc89b3b4221a4b2d95f757a405f" is_pre_election: true
I20260806 07:48:41.096479 26141 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec1137a1dc7429ab1e4378a48a40cdf in term 0.
I20260806 07:48:41.097214 26139 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:48:41.097671 26140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956dbdc89b3b4221a4b2d95f757a405f"
I20260806 07:48:41.103006 26139 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Refusing update from remote peer 5ec1137a1dc7429ab1e4378a48a40cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:48:41.104125 26247 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:48:41.110811 26166 sys_catalog.cc:565] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:41.159754 26282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ec1137a1dc7429ab1e4378a48a40cdf. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:41.160038 26282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:41.159684 26289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:41.160274 26289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:41.195137 26299 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 07:48:41.499372 26280 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:41.499720 26280 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:41.499858 26280 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:41.508503 26280 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:41.508715 26280 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:41.514958 26247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:41.515242 26247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:41.524365 26280 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:41.526306 26280 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:41.528252 26280 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:41.538136 26308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:41.538165 26307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:41.538969 26310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:41.539796 26280 server_base.cc:1061] running on GCE node
I20260806 07:48:41.540405 26280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:41.541580 26280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:41.542824 26280 hybrid_clock.cc:648] HybridClock initialized: now 1786002521542700 us; error 93 us; skew 500 ppm
I20260806 07:48:41.543151 26280 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:41.545650 26280 webserver.cc:533] Webserver started at http://127.20.202.193:33833/ using document root <none> and password file <none>
I20260806 07:48:41.546471 26280 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:41.546587 26280 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:41.546981 26280 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:41.549381 26280 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:41.550278 26280 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:41.556281 26280 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:48:41.559741 26316 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:41.561187 26280 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:48:41.561374 26280 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:41.561640 26280 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:41.589761 26280 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:41.590664 26280 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:41.590958 26280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:41.591936 26280 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:41.593708 26280 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:41.593837 26280 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:41.593971 26280 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:41.594058 26280 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:41.627275 26280 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36401
I20260806 07:48:41.627362 26428 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36401 every 8 connection(s)
I20260806 07:48:41.628726 26280 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:41.633567 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26280
I20260806 07:48:41.633792 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:48:41.638557 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:41.654021 26430 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:41.654547 26430 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:41.654912 26431 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:41.655073 26431 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:41.655853 26431 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:41.655879 26430 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:41.658632 26059 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:41.658818 26130 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:41.660063 26429 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:41.660199 26429 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:41.660717 26429 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:41.662636 26202 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:41.665057 26202 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:40699
I20260806 07:48:41.889298 26265 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:41.889443 26265 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: loading cluster ID for follower catalog manager: success
I20260806 07:48:41.893539 26265 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: acquiring CA information for follower catalog manager: success
I20260806 07:48:41.896327 26265 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:48:42.006459 26439 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:42.006764 26439 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:42.006837 26439 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:42.015344 26439 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:42.015501 26439 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:48:42.025751 26439 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:42.027406 26439 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:42.029345 26439 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:42.039184 26444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:42.039197 26445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:42.040539 26447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:42.040895 26439 server_base.cc:1061] running on GCE node
I20260806 07:48:42.041586 26439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:42.043004 26439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:42.044314 26439 hybrid_clock.cc:648] HybridClock initialized: now 1786002522044134 us; error 140 us; skew 500 ppm
I20260806 07:48:42.044691 26439 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:42.048869 26439 webserver.cc:533] Webserver started at http://127.20.202.194:34059/ using document root <none> and password file <none>
I20260806 07:48:42.050072 26439 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:42.050237 26439 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:42.050750 26439 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:42.054384 26439 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "8cb6a5277a7a4d33a64f4dcbc519ef5e"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:42.055485 26439 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "8cb6a5277a7a4d33a64f4dcbc519ef5e"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:42.063067 26439 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 07:48:42.067621 26453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:42.069782 26439 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.004s
I20260806 07:48:42.070065 26439 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "8cb6a5277a7a4d33a64f4dcbc519ef5e"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:42.070395 26439 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:42.107877 26439 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:42.108846 26439 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:42.109154 26439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:42.110311 26439 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:42.112002 26439 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:42.112140 26439 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:42.112268 26439 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:42.112355 26439 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:42.147773 26439 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:40655
I20260806 07:48:42.147830 26566 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:40655 every 8 connection(s)
I20260806 07:48:42.149339 26439 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:48:42.151010 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26439
I20260806 07:48:42.151266 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:48:42.157469 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:42.171600 26569 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:42.172106 26569 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:42.173231 26569 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:42.174355 26568 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:42.174530 26568 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:42.175073 26568 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:42.175123 26567 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:42.175247 26567 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:42.175544 26130 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:40655)
I20260806 07:48:42.175798 26567 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:42.176956 26059 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:40655)
I20260806 07:48:42.177546 26202 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:40655)
I20260806 07:48:42.178798 26202 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:50493
I20260806 07:48:42.200636 26299 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:42.200790 26299 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: loading cluster ID for follower catalog manager: success
I20260806 07:48:42.205096 26299 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: acquiring CA information for follower catalog manager: success
I20260806 07:48:42.207703 26299 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:48:42.513563 26576 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:42.513985 26576 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:42.514120 26576 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:42.522673 26576 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:42.522900 26576 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:48:42.533439 26576 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:42.535421 26576 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:42.537283 26576 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:42.547293 26583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:42.547328 26582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:42.549767 26585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:42.549988 26576 server_base.cc:1061] running on GCE node
I20260806 07:48:42.550730 26576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:42.551981 26576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:42.553205 26576 hybrid_clock.cc:648] HybridClock initialized: now 1786002522553066 us; error 110 us; skew 500 ppm
I20260806 07:48:42.553551 26576 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:42.556094 26576 webserver.cc:533] Webserver started at http://127.20.202.195:33893/ using document root <none> and password file <none>
I20260806 07:48:42.556962 26576 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:42.557097 26576 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:42.557471 26576 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:48:42.562438 26576 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "ec106cc52581419d95c6160b1c079784"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:42.563261 26576 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "ec106cc52581419d95c6160b1c079784"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:42.569098 26576 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 07:48:42.572834 26591 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:42.574453 26576 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:42.574641 26576 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "ec106cc52581419d95c6160b1c079784"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:42.574916 26576 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:42.594766 26576 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:42.595820 26576 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:42.596113 26576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:42.597167 26576 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:42.599030 26576 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:42.599148 26576 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:42.599277 26576 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:42.599367 26576 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:42.636897 26576 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:37805
I20260806 07:48:42.637012 26705 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:37805 every 8 connection(s)
I20260806 07:48:42.638485 26576 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:48:42.647647 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26576
I20260806 07:48:42.647845 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:48:42.660440 26708 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:42.660565 26706 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:42.660871 26706 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:42.660872 26708 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:42.662180 26708 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:42.662242 26706 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:42.662452 26707 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:42.662611 26707 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:42.663082 26707 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:42.664101 26130 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:37805)
I20260806 07:48:42.664124 26059 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:37805)
I20260806 07:48:42.664173 26202 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:37805)
I20260806 07:48:42.665222 26202 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:33331
I20260806 07:48:42.668499 26429 heartbeater.cc:499] Master 127.20.202.252:36323 was elected leader, sending a full tablet report...
I20260806 07:48:42.673552 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:42.681313 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:42.681495 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26028
I20260806 07:48:42.682359 26089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:42.790333 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26028
I20260806 07:48:42.814349 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--webserver_interface=127.20.202.254
--webserver_port=35491
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
W20260806 07:48:43.090585 26189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39423: connect: Connection refused (error 111)
W20260806 07:48:43.095001 26189 consensus_peers.cc:597] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf -> Peer 39521fcda47b40639ccce746141fd04c (127.20.202.254:39423): Couldn't send request to peer 39521fcda47b40639ccce746141fd04c. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39423: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:48:43.162838 26714 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:43.163244 26714 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:43.163380 26714 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:43.172101 26714 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:43.172233 26714 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:43.172356 26714 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:43.172438 26714 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:48:43.179358 26568 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:39423 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:39423: connect: Connection refused (error 111)
I20260806 07:48:43.181087 26567 heartbeater.cc:499] Master 127.20.202.252:36323 was elected leader, sending a full tablet report...
I20260806 07:48:43.184562 26714 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=35491
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:43.186355 26714 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:43.188261 26714 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:43.196494 26722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:43.196553 26723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:43.197103 26726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:43.203158 26714 server_base.cc:1061] running on GCE node
I20260806 07:48:43.204036 26714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:43.205770 26714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:43.206979 26714 hybrid_clock.cc:648] HybridClock initialized: now 1786002523206828 us; error 118 us; skew 500 ppm
I20260806 07:48:43.207341 26714 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:43.209568 26714 webserver.cc:533] Webserver started at http://127.20.202.254:35491/ using document root <none> and password file <none>
I20260806 07:48:43.210368 26714 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:43.210520 26714 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:43.216039 26714 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:48:43.218873 26732 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:43.220196 26714 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:43.220378 26714 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "39521fcda47b40639ccce746141fd04c"
format_stamp: "Formatted at 2026-08-06 07:48:39 on dist-test-slave-g350"
I20260806 07:48:43.221096 26714 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:43.236424 26714 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:43.237285 26714 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:43.237627 26714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:43.258960 26714 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39423
I20260806 07:48:43.258980 26784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39423 every 8 connection(s)
I20260806 07:48:43.260685 26714 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:43.266086 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26714
I20260806 07:48:43.266426 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26280
I20260806 07:48:43.268539 26785 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:43.271909 26785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap starting.
I20260806 07:48:43.275930 26424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:43.290463 26785 log.cc:826] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:43.299243 26785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:43.299816 26785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap complete.
I20260806 07:48:43.307173 26785 raft_consensus.cc:359] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:43.307863 26785 raft_consensus.cc:740] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39521fcda47b40639ccce746141fd04c, State: Initialized, Role: FOLLOWER
I20260806 07:48:43.308794 26785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:43.309767 26791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:43.310160 26791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:43.310918 26785 sys_catalog.cc:565] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:43.327558 26802 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:43.327675 26802 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: loading cluster ID for follower catalog manager: success
I20260806 07:48:43.332765 26802 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: acquiring CA information for follower catalog manager: success
I20260806 07:48:43.336216 26802 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:43.399756 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26280
I20260806 07:48:43.419860 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36401
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--webserver_port=33833
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:43.654244 26304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:43.655822 26304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:43.667534 26706 heartbeater.cc:499] Master 127.20.202.252:36323 was elected leader, sending a full tablet report...
I20260806 07:48:43.702574 26791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ec1137a1dc7429ab1e4378a48a40cdf. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:43.702956 26791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:43.706233 26749 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ec106cc52581419d95c6160b1c079784" instance_seqno: 1786002522627099) as {username='slave'} at 127.20.202.195:49219; Asking this server to re-register.
I20260806 07:48:43.707201 26707 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:43.709249 26749 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:37805)
W20260806 07:48:43.793073 26808 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:43.793399 26808 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:43.793532 26808 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:43.803017 26808 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:43.803254 26808 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:43.813751 26808 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36401
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=33833
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:43.815450 26808 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:43.817322 26808 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:43.827823 26819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:43.827847 26820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:43.828600 26822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:43.829458 26808 server_base.cc:1061] running on GCE node
I20260806 07:48:43.830204 26808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:43.831390 26808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:43.832607 26808 hybrid_clock.cc:648] HybridClock initialized: now 1786002523832480 us; error 95 us; skew 500 ppm
I20260806 07:48:43.832882 26808 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:43.835135 26808 webserver.cc:533] Webserver started at http://127.20.202.193:33833/ using document root <none> and password file <none>
I20260806 07:48:43.835929 26808 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:43.836035 26808 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:43.841480 26808 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:48:43.844141 26828 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:43.845512 26808 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:43.845729 26808 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:43.846457 26808 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:43.863824 26808 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:43.864712 26808 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:43.865017 26808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:43.866099 26808 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:43.867799 26808 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:43.867910 26808 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:43.868067 26808 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:43.868139 26808 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:43.904510 26808 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36401
I20260806 07:48:43.904605 26942 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36401 every 8 connection(s)
I20260806 07:48:43.906307 26808 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:43.912371 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26808
I20260806 07:48:43.912760 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26808
I20260806 07:48:43.925949 26945 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:43.926348 26945 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:43.927454 26945 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:43.929397 26943 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:43.929423 26938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:43.929572 26943 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:43.930130 26130 ts_manager.cc:194] Re-registered known tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:44.019805 26944 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:44.020139 26944 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:44.020577 26943 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:44.020762 26944 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:44.021441 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26808
I20260806 07:48:44.022147 26749 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:44.023624 26202 ts_manager.cc:194] Re-registered known tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:44.024578 26202 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:57885
W20260806 07:48:44.042387 26736 connection.cc:573] server connection from 127.20.202.193:40113 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:44.042387 26187 connection.cc:573] server connection from 127.20.202.193:57885 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:44.043287 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36401
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--webserver_port=33833
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260806 07:48:44.187382 26568 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:44.188608 26749 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8cb6a5277a7a4d33a64f4dcbc519ef5e" instance_seqno: 1786002522139701) as {username='slave'} at 127.20.202.194:35883; Asking this server to re-register.
I20260806 07:48:44.189257 26568 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:44.190534 26749 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:40655)
I20260806 07:48:44.194589 26810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:44.194962 26810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: LEADER
W20260806 07:48:44.405975 26955 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:44.406322 26955 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:44.406492 26955 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:48:44.406594 26955 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:44.416208 26955 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:44.416474 26955 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:44.428098 26955 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36401
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=33833
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:44.430034 26955 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:44.431964 26955 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:44.442176 26963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:44.442150 26962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:44.443370 26965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:44.443940 26955 server_base.cc:1061] running on GCE node
I20260806 07:48:44.444531 26955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:44.445678 26955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:44.446930 26955 hybrid_clock.cc:648] HybridClock initialized: now 1786002524446765 us; error 115 us; skew 500 ppm
I20260806 07:48:44.447283 26955 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:44.449452 26955 webserver.cc:533] Webserver started at http://127.20.202.193:33833/ using document root <none> and password file <none>
I20260806 07:48:44.450307 26955 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:44.450457 26955 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:44.455884 26955 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 07:48:44.458701 26971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:44.460011 26955 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:44.460191 26955 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:44.460949 26955 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:44.490083 26955 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:44.491041 26955 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:44.491350 26955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:44.492457 26955 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:44.494208 26955 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:44.494331 26955 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:44.494465 26955 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:44.494554 26955 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:44.494947 26955 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:48:44.495076 26955 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:48:44.495164 26955 fault_injection.cc:59] injecting fault for kudu (pid 26955): FLAGS_fault_before_start (process will exit)
I20260806 07:48:44.515756 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36401
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--webserver_port=33833
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:48:44.858665 26979 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:44.858989 26979 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:44.859140 26979 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:44.867738 26979 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:44.867950 26979 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:44.878263 26979 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:43059,127.20.202.213:33097,127.20.202.214:46063,127.20.202.215:36875,127.20.202.216:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36401
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=33833
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:44.879935 26979 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:44.881996 26979 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:44.892164 26985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:44.892122 26984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:44.893137 26987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:44.893621 26979 server_base.cc:1061] running on GCE node
I20260806 07:48:44.894192 26979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:44.895375 26979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:44.896634 26979 hybrid_clock.cc:648] HybridClock initialized: now 1786002524896486 us; error 109 us; skew 500 ppm
I20260806 07:48:44.897013 26979 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:44.899266 26979 webserver.cc:533] Webserver started at http://127.20.202.193:33833/ using document root <none> and password file <none>
I20260806 07:48:44.900051 26979 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:44.900193 26979 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:44.905827 26979 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 07:48:44.908699 26993 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:44.910319 26979 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:44.910507 26979 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:44.911252 26979 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:44.931047 26979 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:44.931921 26979 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:44.932225 26979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:44.933231 26979 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:44.935016 26979 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:44.935122 26979 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:44.935268 26979 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:44.935339 26979 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:44.971233 26979 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36401
I20260806 07:48:44.971289 27106 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36401 every 8 connection(s)
I20260806 07:48:44.972826 26979 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:44.977061 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26979
I20260806 07:48:44.977415 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26979
I20260806 07:48:45.001041 27108 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:45.001624 27108 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:45.002835 27108 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:45.004853 27102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:45.005613 26749 ts_manager.cc:194] Re-registered known tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:45.093086 27107 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:45.093361 27107 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:45.094156 27107 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:45.095168 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26979
I20260806 07:48:45.095995 26202 ts_manager.cc:194] Re-registered known tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:36401)
I20260806 07:48:45.096907 26202 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:33859
W20260806 07:48:45.114957 26187 connection.cc:573] server connection from 127.20.202.193:33859 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:45.115383 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26439
I20260806 07:48:45.123811 26562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:45.211915 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26439
I20260806 07:48:45.230077 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26576
I20260806 07:48:45.238402 26701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:45.324678 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26576
I20260806 07:48:45.343086 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26714
I20260806 07:48:45.349138 26780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:45.455494 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26714
I20260806 07:48:45.478231 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26096
I20260806 07:48:45.479466 26161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:45.586810 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26096
I20260806 07:48:45.609267 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26168
I20260806 07:48:45.610546 26232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:45.719625 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26168
2026-08-06T07:48:45Z chronyd exiting
2026-08-06T07:48:45Z chronyd exiting
2026-08-06T07:48:45Z chronyd exiting
2026-08-06T07:48:45Z chronyd exiting
2026-08-06T07:48:45Z chronyd exiting
2026-08-06T07:48:45Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:45Z Disabled control of system clock
2026-08-06T07:48:45Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:45Z Disabled control of system clock
2026-08-06T07:48:45Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:45Z Disabled control of system clock
2026-08-06T07:48:45Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:45Z Disabled control of system clock
2026-08-06T07:48:45Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:48:45Z Disabled control of system clock
I20260806 07:48:45.922667 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
W20260806 07:48:46.263298 27139 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:46.263621 27139 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:46.263795 27139 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:46.272156 27139 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:46.272254 27139 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:46.272358 27139 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:46.272456 27139 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:46.282779 27139 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:46.284520 27139 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:46.286494 27139 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:46.294922 27144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:46.295181 27145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:46.295678 27147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:46.296731 27139 server_base.cc:1061] running on GCE node
I20260806 07:48:46.297488 27139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:46.299175 27139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:46.300431 27139 hybrid_clock.cc:648] HybridClock initialized: now 1786002526300263 us; error 134 us; skew 500 ppm
I20260806 07:48:46.300809 27139 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:46.303078 27139 webserver.cc:533] Webserver started at http://127.20.202.254:36895/ using document root <none> and password file <none>
I20260806 07:48:46.303803 27139 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:46.303942 27139 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:46.309352 27139 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:46.312323 27153 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:46.313702 27139 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:46.313879 27139 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "39521fcda47b40639ccce746141fd04c"
format_stamp: "Formatted at 2026-08-06 07:48:39 on dist-test-slave-g350"
I20260806 07:48:46.314618 27139 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:46.330557 27139 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:46.331393 27139 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:46.331691 27139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:46.353386 27139 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39423
I20260806 07:48:46.353389 27204 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39423 every 8 connection(s)
I20260806 07:48:46.355134 27139 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:46.361569 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27139
I20260806 07:48:46.361794 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:48:46.362954 27206 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:46.363142 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39911
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
I20260806 07:48:46.368296 27206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap starting.
I20260806 07:48:46.385776 27206 log.cc:826] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:46.394698 27206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:46.395267 27206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap complete.
I20260806 07:48:46.402456 27206 raft_consensus.cc:359] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:46.403178 27206 raft_consensus.cc:740] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39521fcda47b40639ccce746141fd04c, State: Initialized, Role: FOLLOWER
I20260806 07:48:46.403877 27206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:46.404795 27210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:46.405098 27210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:46.405992 27206 sys_catalog.cc:565] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:46.417403 27221 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:46.417520 27221 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: loading cluster ID for follower catalog manager: success
I20260806 07:48:46.422005 27221 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: acquiring CA information for follower catalog manager: success
I20260806 07:48:46.424561 27221 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:46.440146 27139 master_runner.cc:187] Error getting master registration for 127.20.202.253:39911: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39911: connect: Connection refused (error 111),
I20260806 07:48:46.445915 27139 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:46.717237 27208 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:46.717540 27208 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:46.717710 27208 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:46.726231 27208 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:46.726318 27208 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:46.726403 27208 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:46.726492 27208 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:46.737962 27208 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:46.739755 27208 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:46.741824 27208 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:46.750464 27229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:46.750468 27230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:46.751782 27232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:46.752803 27208 server_base.cc:1061] running on GCE node
I20260806 07:48:46.753731 27208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:46.755470 27208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:46.756950 27208 hybrid_clock.cc:648] HybridClock initialized: now 1786002526756779 us; error 125 us; skew 500 ppm
I20260806 07:48:46.757354 27208 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:46.759716 27208 webserver.cc:533] Webserver started at http://127.20.202.253:40151/ using document root <none> and password file <none>
I20260806 07:48:46.760538 27208 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:46.760695 27208 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:46.766619 27208 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:48:46.769865 27238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:46.771524 27208 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:46.771724 27208 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "956dbdc89b3b4221a4b2d95f757a405f"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:46.772477 27208 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:46.811609 27208 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:46.812760 27208 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:46.813112 27208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:46.837628 27208 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:39911
I20260806 07:48:46.837564 27289 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:39911 every 8 connection(s)
I20260806 07:48:46.840040 27208 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:48:46.847656 27290 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:46.848205 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27208
I20260806 07:48:46.848430 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:48:46.849848 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36323
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
I20260806 07:48:46.851310 27290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Bootstrap starting.
I20260806 07:48:46.868481 27290 log.cc:826] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:46.877014 27290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:46.877699 27290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Bootstrap complete.
I20260806 07:48:46.884553 27290 raft_consensus.cc:359] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:46.885281 27290 raft_consensus.cc:740] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956dbdc89b3b4221a4b2d95f757a405f, State: Initialized, Role: FOLLOWER
I20260806 07:48:46.886194 27290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:46.887108 27294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:46.887730 27294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:46.888406 27290 sys_catalog.cc:565] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:46.899981 27305 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:46.900193 27305 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: loading cluster ID for follower catalog manager: success
I20260806 07:48:46.905864 27305 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: acquiring CA information for follower catalog manager: success
I20260806 07:48:46.909279 27305 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:46.939049 27208 master_runner.cc:187] Error getting master registration for 127.20.202.252:36323: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36323: connect: Connection refused (error 111),
I20260806 07:48:46.943158 27208 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:48:47.255861 27292 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:47.256247 27292 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:47.256387 27292 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:47.264930 27292 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:47.265062 27292 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:47.265182 27292 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:47.265265 27292 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:48:47.276226 27292 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:47.278075 27292 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:47.279937 27292 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:47.287940 27314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:47.288733 27313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:47.289690 27316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:47.290860 27292 server_base.cc:1061] running on GCE node
I20260806 07:48:47.291781 27292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:47.293883 27292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:47.297276 27292 hybrid_clock.cc:648] HybridClock initialized: now 1786002527297050 us; error 186 us; skew 500 ppm
I20260806 07:48:47.297771 27292 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:47.300719 27292 webserver.cc:533] Webserver started at http://127.20.202.252:37445/ using document root <none> and password file <none>
I20260806 07:48:47.301740 27292 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:47.301878 27292 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:47.309382 27292 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:48:47.313097 27322 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:47.314859 27292 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 07:48:47.315043 27292 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "5ec1137a1dc7429ab1e4378a48a40cdf"
format_stamp: "Formatted at 2026-08-06 07:48:40 on dist-test-slave-g350"
I20260806 07:48:47.316062 27292 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:47.340166 27292 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:47.341271 27292 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:47.341662 27292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:47.364071 27292 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:36323
I20260806 07:48:47.364121 27375 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:36323 every 8 connection(s)
I20260806 07:48:47.365953 27292 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:48:47.373255 27376 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:47.374945 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27292
I20260806 07:48:47.375164 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:48:47.377033 27376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Bootstrap starting.
I20260806 07:48:47.398746 27376 log.cc:826] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:47.408519 27376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:47.409085 27376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: Bootstrap complete.
I20260806 07:48:47.415925 27376 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:47.416673 27376 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec1137a1dc7429ab1e4378a48a40cdf, State: Initialized, Role: FOLLOWER
I20260806 07:48:47.417284 27376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:47.418092 27381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:47.418327 27381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:47.419108 27376 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:47.430181 27392 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:47.430294 27392 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: loading cluster ID for follower catalog manager: success
I20260806 07:48:47.434918 27392 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: acquiring CA information for follower catalog manager: success
I20260806 07:48:47.438845 27392 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:47.457407 27292 master_runner.cc:264] No leader master found from master 5ec1137a1dc7429ab1e4378a48a40cdf
I20260806 07:48:47.457649 27292 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:48.209240 27395 raft_consensus.cc:493] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:48:48.209636 27395 raft_consensus.cc:515] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:48.211542 27395 leader_election.cc:290] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39521fcda47b40639ccce746141fd04c (127.20.202.254:39423), 5ec1137a1dc7429ab1e4378a48a40cdf (127.20.202.252:36323)
I20260806 07:48:48.212417 27180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "956dbdc89b3b4221a4b2d95f757a405f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39521fcda47b40639ccce746141fd04c" is_pre_election: true
I20260806 07:48:48.212978 27180 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956dbdc89b3b4221a4b2d95f757a405f in term 1.
I20260806 07:48:48.213757 27242 leader_election.cc:304] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39521fcda47b40639ccce746141fd04c, 956dbdc89b3b4221a4b2d95f757a405f; no voters:
I20260806 07:48:48.214214 27395 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:48:48.214380 27395 raft_consensus.cc:493] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:48:48.214538 27395 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:48.219518 27395 raft_consensus.cc:515] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:48.220355 27395 leader_election.cc:290] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [CANDIDATE]: Term 2 election: Requested vote from peers 39521fcda47b40639ccce746141fd04c (127.20.202.254:39423), 5ec1137a1dc7429ab1e4378a48a40cdf (127.20.202.252:36323)
I20260806 07:48:48.220790 27180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "956dbdc89b3b4221a4b2d95f757a405f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39521fcda47b40639ccce746141fd04c"
I20260806 07:48:48.220748 27351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "956dbdc89b3b4221a4b2d95f757a405f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" is_pre_election: true
I20260806 07:48:48.221055 27180 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:48.221053 27350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "956dbdc89b3b4221a4b2d95f757a405f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf"
I20260806 07:48:48.221408 27351 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956dbdc89b3b4221a4b2d95f757a405f in term 1.
I20260806 07:48:48.224989 27180 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956dbdc89b3b4221a4b2d95f757a405f in term 2.
I20260806 07:48:48.225441 27242 leader_election.cc:304] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39521fcda47b40639ccce746141fd04c, 956dbdc89b3b4221a4b2d95f757a405f; no voters: 5ec1137a1dc7429ab1e4378a48a40cdf
I20260806 07:48:48.225785 27395 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:48:48.226315 27395 raft_consensus.cc:697] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [term 2 LEADER]: Becoming Leader. State: Replica: 956dbdc89b3b4221a4b2d95f757a405f, State: Running, Role: LEADER
I20260806 07:48:48.226799 27395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:48.230304 27398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 956dbdc89b3b4221a4b2d95f757a405f. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:48.230563 27398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:48.231245 27400 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:48:48.233176 27400 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:48:48.234072 27400 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:48.234175 27400 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:48:48.235293 27400 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:48:48.236083 27400 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f: Loaded TSK: 0
I20260806 07:48:48.236963 27400 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:48:48.275879 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:48:48.623991 27401 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:48.624343 27401 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:48.624490 27401 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:48.632870 27401 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:48.633093 27401 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:48.643077 27401 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:48.644783 27401 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:48.646663 27401 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:48.656702 27406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:48.656713 27407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:48.658186 27409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:48.658468 27401 server_base.cc:1061] running on GCE node
I20260806 07:48:48.659024 27401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:48.660158 27401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:48.661361 27401 hybrid_clock.cc:648] HybridClock initialized: now 1786002528661241 us; error 86 us; skew 500 ppm
I20260806 07:48:48.661743 27401 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:48.663956 27401 webserver.cc:533] Webserver started at http://127.20.202.193:40077/ using document root <none> and password file <none>
I20260806 07:48:48.664737 27401 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:48.664876 27401 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:48.670349 27401 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:48.672994 27415 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:48.674403 27401 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:48.674578 27401 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:48.675289 27401 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:48.690791 27401 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:48.691632 27401 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:48.691947 27401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:48.693002 27401 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:48.694671 27401 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:48.694749 27401 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:48.694830 27401 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:48.694885 27401 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:48.731010 27401 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35769
I20260806 07:48:48.731040 27531 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35769 every 8 connection(s)
I20260806 07:48:48.732622 27401 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:48.738409 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27401
I20260806 07:48:48.738658 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:48:48.744565 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:48.745684 27350 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:48:48.752812 27350 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [term 2 FOLLOWER]: Refusing update from remote peer 956dbdc89b3b4221a4b2d95f757a405f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:48:48.754506 27542 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:48:48.759032 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 956dbdc89b3b4221a4b2d95f757a405f. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:48.759884 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:48.762773 27532 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:48.762919 27533 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:48.763247 27533 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:48.764542 27533 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:48.763243 27532 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:48.766393 27532 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:48.768638 27534 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:48.768790 27534 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:48.768745 27340 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:35769)
I20260806 07:48:48.768743 27170 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:35769)
I20260806 07:48:48.769326 27534 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:48.773032 27255 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:35769)
I20260806 07:48:48.773432 27180 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 2 FOLLOWER]: Refusing update from remote peer 956dbdc89b3b4221a4b2d95f757a405f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:48:48.774441 27542 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [LEADER]: Connected to new peer: Peer: permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:48:48.775893 27255 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:42803
I20260806 07:48:48.777173 27542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:48.777462 27542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:48.787493 27553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:48.787935 27553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:48.788419 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:48.788636 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec1137a1dc7429ab1e4378a48a40cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:48.788324 27549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 956dbdc89b3b4221a4b2d95f757a405f. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:48.789265 27549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:48:49.108588 27543 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:49.108906 27543 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:49.108995 27543 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:49.117754 27543 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:49.117913 27543 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:48:49.128346 27543 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:49.129988 27543 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:49.131714 27543 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:49.141301 27560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:49.141623 27562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:49.141804 27559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:49.142617 27543 server_base.cc:1061] running on GCE node
I20260806 07:48:49.143160 27543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:49.144191 27543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:49.145397 27543 hybrid_clock.cc:648] HybridClock initialized: now 1786002529145256 us; error 110 us; skew 500 ppm
I20260806 07:48:49.145752 27543 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:49.147763 27543 webserver.cc:533] Webserver started at http://127.20.202.194:36525/ using document root <none> and password file <none>
I20260806 07:48:49.148536 27543 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:49.148645 27543 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:49.154109 27543 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:48:49.156721 27568 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:49.158264 27543 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:49.158412 27543 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "8cb6a5277a7a4d33a64f4dcbc519ef5e"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:49.159133 27543 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:49.200884 27543 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:49.201815 27543 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:49.202076 27543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:49.203029 27543 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:49.204741 27543 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:49.204826 27543 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:49.204913 27543 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:49.204967 27543 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:49.240657 27543 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:38187
I20260806 07:48:49.240720 27683 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:38187 every 8 connection(s)
I20260806 07:48:49.242265 27543 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:48:49.244449 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27543
I20260806 07:48:49.244654 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:48:49.246682 27546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:49.247089 27546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:49.250116 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:49.259552 27684 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:49.259979 27684 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:49.261067 27684 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:49.263049 27340 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:38187)
I20260806 07:48:49.266139 27685 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:49.266173 27686 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:49.266320 27685 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:49.266384 27686 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:49.266891 27686 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:49.266973 27685 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:49.267930 27255 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:38187)
I20260806 07:48:49.268159 27170 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:38187)
I20260806 07:48:49.268922 27255 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:51219
W20260806 07:48:49.603876 27691 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:49.604176 27691 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:49.604339 27691 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:49.612870 27691 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:49.613078 27691 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:48:49.624327 27691 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:49.626027 27691 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:49.627903 27691 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:49.638018 27696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:49.638021 27697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:49.638624 27699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:49.639113 27691 server_base.cc:1061] running on GCE node
I20260806 07:48:49.639685 27691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:49.640795 27691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:49.642037 27691 hybrid_clock.cc:648] HybridClock initialized: now 1786002529641903 us; error 101 us; skew 500 ppm
I20260806 07:48:49.642364 27691 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:49.644546 27691 webserver.cc:533] Webserver started at http://127.20.202.195:42737/ using document root <none> and password file <none>
I20260806 07:48:49.645300 27691 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:49.645463 27691 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:49.650890 27691 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 07:48:49.653725 27705 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:49.655135 27691 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:49.655335 27691 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "ec106cc52581419d95c6160b1c079784"
format_stamp: "Formatted at 2026-08-06 07:48:42 on dist-test-slave-g350"
I20260806 07:48:49.656112 27691 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:49.682767 27691 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:49.683691 27691 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:49.684002 27691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:49.685001 27691 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:49.686936 27691 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:49.687014 27691 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:49.687146 27691 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:49.687196 27691 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:49.721318 27691 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:42879
I20260806 07:48:49.721372 27820 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:42879 every 8 connection(s)
I20260806 07:48:49.722983 27691 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:48:49.728471 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27691
I20260806 07:48:49.728701 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:48:49.745501 27821 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:49.745626 27822 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:49.745987 27822 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:49.745985 27821 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:49.747264 27822 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:49.747292 27821 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:49.749464 27170 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:42879)
I20260806 07:48:49.749966 27340 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:42879)
I20260806 07:48:49.751547 27823 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:49.751709 27823 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:49.752250 27823 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:49.753278 27255 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:42879)
I20260806 07:48:49.754289 27255 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:53937
I20260806 07:48:49.755692 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:49.762842 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:48:49.763034 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27139
I20260806 07:48:49.763903 27200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:49.779786 27534 heartbeater.cc:499] Master 127.20.202.253:39911 was elected leader, sending a full tablet report...
I20260806 07:48:49.873355 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27139
I20260806 07:48:49.897718 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--webserver_interface=127.20.202.254
--webserver_port=36895
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323 with env {}
W20260806 07:48:50.254418 27829 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:50.254715 27829 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:50.254873 27829 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:50.263760 27829 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:48:50.263854 27829 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:50.263965 27829 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:48:50.264050 27829 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:48:50.270390 27685 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:39423 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:39423: connect: Connection refused (error 111)
I20260806 07:48:50.271773 27686 heartbeater.cc:499] Master 127.20.202.253:39911 was elected leader, sending a full tablet report...
I20260806 07:48:50.275455 27829 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=36895
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:50.277124 27829 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:50.279147 27829 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:50.288426 27836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:50.288416 27835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:50.288683 27838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:50.289662 27829 server_base.cc:1061] running on GCE node
I20260806 07:48:50.290531 27829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:50.292240 27829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:50.293491 27829 hybrid_clock.cc:648] HybridClock initialized: now 1786002530293333 us; error 121 us; skew 500 ppm
I20260806 07:48:50.293853 27829 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:50.295944 27829 webserver.cc:533] Webserver started at http://127.20.202.254:36895/ using document root <none> and password file <none>
I20260806 07:48:50.296649 27829 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:50.296741 27829 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:50.302243 27829 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:48:50.305127 27845 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:50.306668 27829 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:48:50.306852 27829 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "39521fcda47b40639ccce746141fd04c"
format_stamp: "Formatted at 2026-08-06 07:48:39 on dist-test-slave-g350"
I20260806 07:48:50.307602 27829 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260806 07:48:50.318131 27242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39423: connect: Connection refused (error 111)
W20260806 07:48:50.319736 27242 consensus_peers.cc:597] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f -> Peer 39521fcda47b40639ccce746141fd04c (127.20.202.254:39423): Couldn't send request to peer 39521fcda47b40639ccce746141fd04c. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39423: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:48:50.321972 27829 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:50.322772 27829 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:50.323056 27829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:50.345924 27900 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39423 every 8 connection(s)
I20260806 07:48:50.345980 27829 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39423
I20260806 07:48:50.348591 27829 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:48:50.350747 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27829
I20260806 07:48:50.351110 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27401
I20260806 07:48:50.361608 27901 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:48:50.367556 27901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap starting.
I20260806 07:48:50.378517 27527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:50.395601 27901 log.cc:826] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:48:50.408988 27901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:48:50.409771 27901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: Bootstrap complete.
I20260806 07:48:50.422533 27901 raft_consensus.cc:359] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:50.423806 27901 raft_consensus.cc:740] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39521fcda47b40639ccce746141fd04c, State: Initialized, Role: FOLLOWER
I20260806 07:48:50.424786 27901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } }
I20260806 07:48:50.425812 27907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:50.426186 27907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:48:50.427311 27901 sys_catalog.cc:565] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:48:50.461769 27918 catalog_manager.cc:1295] Loaded cluster ID: b3c7a10a37e6448dabd96cecd8829728
I20260806 07:48:50.461951 27918 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: loading cluster ID for follower catalog manager: success
I20260806 07:48:50.465790 27918 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: acquiring CA information for follower catalog manager: success
I20260806 07:48:50.468354 27918 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:48:50.506425 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27401
I20260806 07:48:50.529824 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35769
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--webserver_port=40077
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:48:50.756842 27823 heartbeater.cc:499] Master 127.20.202.253:39911 was elected leader, sending a full tablet report...
I20260806 07:48:50.763471 27865 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ec106cc52581419d95c6160b1c079784" instance_seqno: 1786002529713036) as {username='slave'} at 127.20.202.195:59235; Asking this server to re-register.
I20260806 07:48:50.764348 27822 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:50.766170 27865 ts_manager.cc:194] Registered new tserver with Master: ec106cc52581419d95c6160b1c079784 (127.20.202.195:42879)
I20260806 07:48:50.865095 27542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:50.865386 27542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: LEADER
I20260806 07:48:50.900245 27907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 956dbdc89b3b4221a4b2d95f757a405f. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:50.900588 27907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39521fcda47b40639ccce746141fd04c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:48:50.909883 27924 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:50.910203 27924 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:50.910343 27924 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:50.919268 27924 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:50.919471 27924 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:50.930343 27924 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35769
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40077
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:50.932042 27924 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:50.934012 27924 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:50.944151 27935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:50.944125 27934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:50.945864 27937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:50.946862 27924 server_base.cc:1061] running on GCE node
I20260806 07:48:50.947448 27924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:50.948598 27924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:50.951987 27924 hybrid_clock.cc:648] HybridClock initialized: now 1786002530951819 us; error 131 us; skew 500 ppm
I20260806 07:48:50.952319 27924 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:50.954582 27924 webserver.cc:533] Webserver started at http://127.20.202.193:40077/ using document root <none> and password file <none>
I20260806 07:48:50.955405 27924 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:50.955540 27924 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:50.961292 27924 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:48:50.964296 27943 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:50.965812 27924 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:48:50.965984 27924 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:50.966727 27924 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:51.009528 27924 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:51.010823 27924 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:51.011183 27924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:51.012751 27924 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:51.014665 27924 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:51.014772 27924 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:51.014909 27924 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:51.014998 27924 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:51.052045 27924 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35769
I20260806 07:48:51.052134 28058 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35769 every 8 connection(s)
I20260806 07:48:51.053660 27924 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:51.054065 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27924
I20260806 07:48:51.054406 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27924
I20260806 07:48:51.072744 28054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:51.161937 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27924
W20260806 07:48:51.180071 27849 connection.cc:573] server connection from 127.20.202.193:48841 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:51.180073 27326 connection.cc:573] server connection from 127.20.202.193:54081 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:51.180924 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35769
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--webserver_port=40077
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
W20260806 07:48:51.181318 28068 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:48:51.061371 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:49677 (local address 127.20.202.253:39911)
0806 07:48:51.061898 (+ 527us) server_negotiation.cc:207] Beginning negotiation
0806 07:48:51.061911 (+ 13us) server_negotiation.cc:400] Waiting for connection header
0806 07:48:51.070645 (+ 8734us) server_negotiation.cc:408] Connection header received
0806 07:48:51.070733 (+ 88us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:48:51.070741 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:48:51.070827 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:48:51.070991 (+ 164us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:48:51.071991 (+ 1000us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:48:51.073291 (+ 1300us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:48:51.161395 (+ 88104us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:48:51.162033 (+ 638us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:48:51.180234 (+ 18201us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:49677: BlockingRecv error: recv got EOF from 127.20.202.193:49677 (error 108)
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":304,"threads_started":1}
I20260806 07:48:51.278617 27685 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:51.279707 27865 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8cb6a5277a7a4d33a64f4dcbc519ef5e" instance_seqno: 1786002529232220) as {username='slave'} at 127.20.202.194:53565; Asking this server to re-register.
I20260806 07:48:51.280346 27685 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:51.281679 27865 ts_manager.cc:194] Registered new tserver with Master: 8cb6a5277a7a4d33a64f4dcbc519ef5e (127.20.202.194:38187)
I20260806 07:48:51.467463 27926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "956dbdc89b3b4221a4b2d95f757a405f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39521fcda47b40639ccce746141fd04c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39423 } } peers { permanent_uuid: "956dbdc89b3b4221a4b2d95f757a405f" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39911 } } peers { permanent_uuid: "5ec1137a1dc7429ab1e4378a48a40cdf" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36323 } } }
I20260806 07:48:51.467819 27926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 956dbdc89b3b4221a4b2d95f757a405f [sys.catalog]: This master's current role is: LEADER
W20260806 07:48:51.529647 28071 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:51.530005 28071 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:51.530099 28071 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:48:51.530160 28071 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:51.538615 28071 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:51.538801 28071 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:51.549507 28071 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35769
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40077
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:51.551323 28071 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:51.553144 28071 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:51.563725 28081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:51.563863 28080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:51.563689 28083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:51.565471 28071 server_base.cc:1061] running on GCE node
I20260806 07:48:51.566152 28071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:51.567404 28071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:51.568686 28071 hybrid_clock.cc:648] HybridClock initialized: now 1786002531568534 us; error 98 us; skew 500 ppm
I20260806 07:48:51.569043 28071 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:51.571509 28071 webserver.cc:533] Webserver started at http://127.20.202.193:40077/ using document root <none> and password file <none>
I20260806 07:48:51.572335 28071 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:51.572543 28071 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:51.578902 28071 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:48:51.581877 28089 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:51.583318 28071 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:48:51.583528 28071 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:51.584378 28071 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:51.600714 28071 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:51.601584 28071 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:51.601933 28071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:51.602917 28071 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:51.604592 28071 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:51.604702 28071 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:51.604830 28071 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:51.604923 28071 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:48:51.605280 28071 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:48:51.605499 28071 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:48:51.605578 28071 fault_injection.cc:59] injecting fault for kudu (pid 28071): FLAGS_fault_before_start (process will exit)
I20260806 07:48:51.623003 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35769
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--webserver_port=40077
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:48:51.968492 28097 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:48:51.968783 28097 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:48:51.968922 28097 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:48:51.978336 28097 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:48:51.978538 28097 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:48:51.988583 28097 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34303,127.20.202.213:40211,127.20.202.214:43209,127.20.202.215:39441,127.20.202.216:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35769
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40077
--tserver_master_addrs=127.20.202.254:39423,127.20.202.253:39911,127.20.202.252:36323
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:48:51.990343 28097 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:48:51.992278 28097 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:48:52.002499 28102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:52.002526 28103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:48:52.003083 28105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:48:52.004052 28097 server_base.cc:1061] running on GCE node
I20260806 07:48:52.004634 28097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:48:52.005820 28097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:48:52.007064 28097 hybrid_clock.cc:648] HybridClock initialized: now 1786002532006905 us; error 130 us; skew 500 ppm
I20260806 07:48:52.007407 28097 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:48:52.009574 28097 webserver.cc:533] Webserver started at http://127.20.202.193:40077/ using document root <none> and password file <none>
I20260806 07:48:52.010390 28097 fs_manager.cc:362] Metadata directory not provided
I20260806 07:48:52.010524 28097 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:48:52.017004 28097 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:48:52.019860 28111 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:52.021286 28097 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:48:52.021464 28097 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "8bef034c974745f3b26d8e5a9f2fba03"
format_stamp: "Formatted at 2026-08-06 07:48:41 on dist-test-slave-g350"
I20260806 07:48:52.022279 28097 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:48:52.035336 28097 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:48:52.036247 28097 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:48:52.036536 28097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:48:52.037459 28097 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:48:52.039150 28097 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:48:52.039253 28097 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:52.039404 28097 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:48:52.039476 28097 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:48:52.074893 28097 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35769
I20260806 07:48:52.074930 28225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35769 every 8 connection(s)
I20260806 07:48:52.076440 28097 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:48:52.083930 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28097
I20260806 07:48:52.084300 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28097
I20260806 07:48:52.096935 28226 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36323
I20260806 07:48:52.098387 28227 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39423
I20260806 07:48:52.098548 28227 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:52.098677 28228 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39911
I20260806 07:48:52.098702 28226 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:52.098836 28228 heartbeater.cc:461] Registering TS with master...
I20260806 07:48:52.099828 28227 heartbeater.cc:507] Master 127.20.202.254:39423 requested a full tablet report, sending...
I20260806 07:48:52.099869 28228 heartbeater.cc:507] Master 127.20.202.253:39911 requested a full tablet report, sending...
I20260806 07:48:52.099915 28226 heartbeater.cc:507] Master 127.20.202.252:36323 requested a full tablet report, sending...
I20260806 07:48:52.102108 27255 ts_manager.cc:194] Re-registered known tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:35769)
I20260806 07:48:52.102584 27340 ts_manager.cc:194] Re-registered known tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:35769)
I20260806 07:48:52.103178 27255 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:51663
I20260806 07:48:52.103295 27865 ts_manager.cc:194] Registered new tserver with Master: 8bef034c974745f3b26d8e5a9f2fba03 (127.20.202.193:35769)
I20260806 07:48:52.110771 28221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:52.203522 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28097
I20260806 07:48:52.222903 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27543
I20260806 07:48:52.231003 27679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:52.318003 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27543
W20260806 07:48:52.336628 27846 connection.cc:573] server connection from 127.20.202.194:53565 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:48:52.336645 27325 connection.cc:573] server connection from 127.20.202.194:37559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:48:52.337226 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27691
I20260806 07:48:52.345425 27816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:52.433265 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27691
I20260806 07:48:52.451499 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27829
I20260806 07:48:52.458602 27896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:52.563668 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27829
I20260806 07:48:52.585937 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27208
W20260806 07:48:52.586727 28245 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:48:52.464480 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.20.202.254:39423 (local address 127.0.0.1:38186)
0806 07:48:52.465087 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 07:48:52.465108 (+ 21us) client_negotiation.cc:175] Beginning negotiation
0806 07:48:52.465233 (+ 125us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0806 07:48:52.563748 (+ 98515us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0806 07:48:52.563755 (+ 7us) client_negotiation.cc:404] Received NEGOTIATE response from server
0806 07:48:52.564530 (+ 775us) client_negotiation.cc:190] Negotiated authn=CERTIFICATE
0806 07:48:52.565205 (+ 675us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0806 07:48:52.565219 (+ 14us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0806 07:48:52.585850 (+ 20631us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39423: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":307,"threads_started":1}
I20260806 07:48:52.587241 27285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:52.587095 27292 master_runner.cc:187] Error getting master registration for 127.20.202.254:39423: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39423: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260806 07:48:52.592525 27292 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:48:52.697993 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27208
I20260806 07:48:52.725819 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27292
I20260806 07:48:52.727061 27371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:48:52.833580 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27292
2026-08-06T07:48:52Z chronyd exiting
2026-08-06T07:48:52Z chronyd exiting
2026-08-06T07:48:52Z chronyd exiting
2026-08-06T07:48:52Z chronyd exiting
2026-08-06T07:48:52Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/2 (13620 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/3
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:49:00: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:49:08.841199 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--webserver_interface=127.20.202.254
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
W20260806 07:49:09.178550 28530 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:09.178884 28530 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:09.179031 28530 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:09.187316 28530 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:09.187399 28530 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:09.187495 28530 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:09.187587 28530 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:09.197413 28530 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:09.199045 28530 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:09.200860 28530 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:09.209343 28536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:09.209333 28537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:09.210462 28539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:09.210971 28530 server_base.cc:1061] running on GCE node
I20260806 07:49:09.211790 28530 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:09.211992 28530 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:09.212100 28530 hybrid_clock.cc:648] HybridClock initialized: now 1786002549212070 us; error 0 us; skew 500 ppm
I20260806 07:49:09.212447 28530 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:09.215121 28530 webserver.cc:533] Webserver started at http://127.20.202.254:39709/ using document root <none> and password file <none>
I20260806 07:49:09.215902 28530 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:09.216046 28530 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:09.216441 28530 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:09.218868 28530 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "556b9231464f451f80620b7995c6b2d0"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:09.219678 28530 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "556b9231464f451f80620b7995c6b2d0"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:09.226217 28530 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 07:49:09.229797 28544 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:09.231154 28530 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:49:09.231379 28530 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "556b9231464f451f80620b7995c6b2d0"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:09.231613 28530 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:09.248818 28530 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:09.249763 28530 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:09.250048 28530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:09.271337 28530 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:42007
I20260806 07:49:09.271359 28596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:42007 every 8 connection(s)
I20260806 07:49:09.272915 28530 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:49:09.277359 28597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:49:09.282950 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28530
I20260806 07:49:09.283159 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:49:09.283373 28597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:09.284237 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34181
--webserver_interface=127.20.202.253
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
I20260806 07:49:09.302873 28597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:49:09.303994 28545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111)
W20260806 07:49:09.305627 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34181: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111)
I20260806 07:49:09.354970 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } attempt: 1
W20260806 07:49:09.357098 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34181: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111)
I20260806 07:49:09.425374 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } attempt: 2
W20260806 07:49:09.427798 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34181: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111)
I20260806 07:49:09.519098 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } attempt: 3
W20260806 07:49:09.521384 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34181: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111)
W20260806 07:49:09.621997 28601 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:09.622366 28601 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:09.622510 28601 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:09.630954 28601 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:09.631079 28601 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:09.631207 28601 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:09.631290 28601 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:09.641530 28601 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:09.643234 28601 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:09.645090 28601 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:09.653265 28609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:09.653292 28608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:09.654227 28611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:09.654529 28601 server_base.cc:1061] running on GCE node
I20260806 07:49:09.655318 28601 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:09.655757 28601 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:09.655872 28601 hybrid_clock.cc:648] HybridClock initialized: now 1786002549655832 us; error 0 us; skew 500 ppm
I20260806 07:49:09.656278 28601 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:09.659104 28601 webserver.cc:533] Webserver started at http://127.20.202.253:41161/ using document root <none> and password file <none>
I20260806 07:49:09.659888 28601 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:09.660018 28601 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:09.660385 28601 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:09.662832 28601 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "6a09d3451a5c4d0aa118905164318b9b"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:09.663652 28601 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "6a09d3451a5c4d0aa118905164318b9b"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:09.664660 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } attempt: 4
W20260806 07:49:09.666793 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34181: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111)
I20260806 07:49:09.669723 28601 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 07:49:09.673051 28616 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:09.674530 28601 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:09.674713 28601 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "6a09d3451a5c4d0aa118905164318b9b"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:09.674937 28601 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:09.698225 28601 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:09.699132 28601 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:09.699381 28601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:09.719429 28601 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:34181
I20260806 07:49:09.719450 28667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:34181 every 8 connection(s)
I20260806 07:49:09.720963 28601 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:49:09.725456 28668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:49:09.727105 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28601
I20260806 07:49:09.727339 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:49:09.728441 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:43789
--webserver_interface=127.20.202.252
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
I20260806 07:49:09.731704 28668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:09.745851 28668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:09.756124 28668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:49:09.757505 28620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
W20260806 07:49:09.759279 28668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
I20260806 07:49:09.808671 28668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 1
W20260806 07:49:09.811195 28668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
I20260806 07:49:09.879446 28668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 2
W20260806 07:49:09.881685 28668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
I20260806 07:49:09.966061 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } attempt: 5
I20260806 07:49:09.972955 28668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 3
I20260806 07:49:09.974236 28597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:49:09.974951 28668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
W20260806 07:49:09.976495 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
I20260806 07:49:10.027753 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 1
W20260806 07:49:10.030202 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
W20260806 07:49:10.084724 28670 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:10.085052 28670 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:10.085117 28670 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:10.093317 28670 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:10.093410 28670 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:10.093467 28670 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:10.093523 28670 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:10.098465 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 2
W20260806 07:49:10.100625 28597 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
I20260806 07:49:10.104341 28670 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:43789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:10.106015 28670 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:10.107826 28670 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:10.115569 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:10.115716 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:10.116302 28682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:10.117210 28670 server_base.cc:1061] running on GCE node
I20260806 07:49:10.117990 28670 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:10.118167 28670 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:10.118266 28670 hybrid_clock.cc:648] HybridClock initialized: now 1786002550118242 us; error 0 us; skew 500 ppm
I20260806 07:49:10.118185 28668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 4
I20260806 07:49:10.118641 28670 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260806 07:49:10.120144 28668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111)
I20260806 07:49:10.121398 28670 webserver.cc:533] Webserver started at http://127.20.202.252:38005/ using document root <none> and password file <none>
I20260806 07:49:10.122207 28670 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:10.122336 28670 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:10.122754 28670 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:10.125154 28670 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "28bee9a34538401481754941e47058fa"
format_stamp: "Formatted at 2026-08-06 07:49:10 on dist-test-slave-g350"
I20260806 07:49:10.126029 28670 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "28bee9a34538401481754941e47058fa"
format_stamp: "Formatted at 2026-08-06 07:49:10 on dist-test-slave-g350"
I20260806 07:49:10.131357 28670 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:49:10.134668 28687 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:10.135938 28670 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:10.136132 28670 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "28bee9a34538401481754941e47058fa"
format_stamp: "Formatted at 2026-08-06 07:49:10 on dist-test-slave-g350"
I20260806 07:49:10.136435 28670 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:10.166898 28670 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:10.167865 28670 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:10.168136 28670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:10.189456 28670 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:43789
I20260806 07:49:10.189481 28739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:43789 every 8 connection(s)
I20260806 07:49:10.191239 28670 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:49:10.193346 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28670
I20260806 07:49:10.193576 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:49:10.195930 28740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:49:10.204331 28740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:10.206856 28597 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 3
I20260806 07:49:10.218240 28740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:10.223850 28597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap starting.
I20260806 07:49:10.227249 28597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Neither blocks nor log segments found. Creating new log.
I20260806 07:49:10.228459 28740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:10.228600 28597 log.cc:826] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:10.231935 28597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: No bootstrap required, opened a new log
I20260806 07:49:10.237792 28597 raft_consensus.cc:359] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.238128 28597 raft_consensus.cc:385] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:49:10.238262 28597 raft_consensus.cc:740] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556b9231464f451f80620b7995c6b2d0, State: Initialized, Role: FOLLOWER
I20260806 07:49:10.238530 28740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Bootstrap starting.
I20260806 07:49:10.238966 28597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.240120 28747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.240458 28747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.241051 28597 sys_catalog.cc:565] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:10.242295 28740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Neither blocks nor log segments found. Creating new log.
I20260806 07:49:10.243605 28740 log.cc:826] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:10.246763 28740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: No bootstrap required, opened a new log
I20260806 07:49:10.251662 28747 raft_consensus.cc:493] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:49:10.251904 28747 raft_consensus.cc:515] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
W20260806 07:49:10.252712 28550 tablet.cc:2406] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:49:10.252843 28740 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.253171 28740 raft_consensus.cc:385] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:49:10.253293 28740 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bee9a34538401481754941e47058fa, State: Initialized, Role: FOLLOWER
I20260806 07:49:10.254043 28740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.255514 28747 leader_election.cc:290] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a09d3451a5c4d0aa118905164318b9b (127.20.202.253:34181), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:10.258551 28761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.259105 28740 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:10.259132 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa" is_pre_election: true
I20260806 07:49:10.259652 28715 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 556b9231464f451f80620b7995c6b2d0 in term 0.
I20260806 07:49:10.261248 28548 leader_election.cc:304] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bee9a34538401481754941e47058fa, 556b9231464f451f80620b7995c6b2d0; no voters:
W20260806 07:49:10.262740 28759 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:49:10.262876 28759 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:49:10.263185 28747 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:49:10.263339 28747 raft_consensus.cc:493] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:49:10.263448 28747 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:49:10.266836 28747 raft_consensus.cc:515] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.268142 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa"
I20260806 07:49:10.268334 28715 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:49:10.272791 28715 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 556b9231464f451f80620b7995c6b2d0 in term 1.
I20260806 07:49:10.273754 28548 leader_election.cc:304] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bee9a34538401481754941e47058fa, 556b9231464f451f80620b7995c6b2d0; no voters:
I20260806 07:49:10.274662 28747 leader_election.cc:290] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6a09d3451a5c4d0aa118905164318b9b (127.20.202.253:34181), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:10.275043 28747 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:49:10.275207 28747 raft_consensus.cc:697] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 556b9231464f451f80620b7995c6b2d0, State: Running, Role: LEADER
I20260806 07:49:10.275725 28747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.263197 28530 master_runner.cc:187] Error getting master registration for 127.20.202.253:34181: OK, instance_id { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" instance_seqno: 1786002549714208 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:49:10.279397 28530 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:10.280514 28761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.288309 28766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556b9231464f451f80620b7995c6b2d0. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.288714 28766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:10.302275 28771 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:49:10.307922 28771 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:49:10.322132 28715 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 1 FOLLOWER]: Refusing update from remote peer 556b9231464f451f80620b7995c6b2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:49:10.323606 28766 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 07:49:10.323787 28775 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:49:10.324095 28775 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:49:10.331393 28670 master_runner.cc:187] Error getting master registration for 127.20.202.253:34181: OK, instance_id { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" instance_seqno: 1786002549714208 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:49:10.341217 28670 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:10.360446 28761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556b9231464f451f80620b7995c6b2d0. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.360700 28761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.362210 28776 mvcc.cc:204] Tried to move back new op lower bound from 7315466446102925312 to 7315466445934608384. Current Snapshot: MvccSnapshot[applied={T|T < 7315466446102925312}]
I20260806 07:49:10.363269 28766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.363571 28766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:10.367642 28771 catalog_manager.cc:1383] Generated new cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:10.367754 28771 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:49:10.390480 28761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.390724 28761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.411096 28771 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:49:10.413353 28771 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:49:10.419387 28668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } attempt: 5
I20260806 07:49:10.438140 28668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Bootstrap starting.
I20260806 07:49:10.445894 28668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Neither blocks nor log segments found. Creating new log.
I20260806 07:49:10.446182 28771 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Generated new TSK 0
I20260806 07:49:10.447314 28771 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:49:10.448779 28668 log.cc:826] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:10.459762 28668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: No bootstrap required, opened a new log
I20260806 07:49:10.467947 28668 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.468405 28668 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:49:10.468575 28668 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a09d3451a5c4d0aa118905164318b9b, State: Initialized, Role: FOLLOWER
I20260806 07:49:10.468691 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--time_source=system_unsync with env {}
I20260806 07:49:10.469770 28668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:10.476917 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.479372 28643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a09d3451a5c4d0aa118905164318b9b" is_pre_election: true
I20260806 07:49:10.479803 28641 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:49:10.479732 28668 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:10.482239 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.483731 28641 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 1 FOLLOWER]: Refusing update from remote peer 556b9231464f451f80620b7995c6b2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:49:10.483898 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a09d3451a5c4d0aa118905164318b9b"
I20260806 07:49:10.486404 28766 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:49:10.530544 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556b9231464f451f80620b7995c6b2d0. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.530825 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.531337 28793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.532547 28793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:10.552795 28800 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 07:49:10.962494 28784 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:10.962814 28784 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:10.962949 28784 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:10.971215 28784 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:10.971410 28784 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:10.981792 28784 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:10.983726 28784 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:10.985908 28784 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:49:10.990993 28766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:10.991335 28766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: LEADER
W20260806 07:49:10.997501 28808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:10.997713 28811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:10.997502 28809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:10.998847 28784 server_base.cc:1061] running on GCE node
I20260806 07:49:10.999460 28784 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:10.999607 28784 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:10.999702 28784 hybrid_clock.cc:648] HybridClock initialized: now 1786002550999656 us; error 0 us; skew 500 ppm
I20260806 07:49:10.999989 28784 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:11.002388 28784 webserver.cc:533] Webserver started at http://127.20.202.193:44407/ using document root <none> and password file <none>
I20260806 07:49:11.003104 28784 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:11.003209 28784 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:11.003492 28784 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:11.005913 28784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:11.006644 28784 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:11.011735 28784 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:49:11.015112 28816 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:11.016374 28784 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:49:11.016566 28784 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:11.016773 28784 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:11.040031 28784 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:11.041025 28784 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:11.041383 28784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:11.042546 28784 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:11.044310 28784 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:11.044435 28784 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:11.044589 28784 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:11.044700 28784 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:11.079555 28784 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38255
I20260806 07:49:11.079604 28930 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38255 every 8 connection(s)
I20260806 07:49:11.081382 28784 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:11.084322 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28784
I20260806 07:49:11.084540 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:49:11.093447 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--time_source=system_unsync with env {}
I20260806 07:49:11.121905 28932 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:11.122340 28932 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:11.123342 28932 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:11.125176 28931 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:11.125334 28931 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:11.125885 28931 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:11.125865 28562 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:11.127658 28704 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:11.128130 28562 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:54935
I20260806 07:49:11.133327 28933 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:11.133492 28933 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:11.133854 28933 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:11.135377 28633 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:11.327323 28775 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:11.327466 28775 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: loading cluster ID for follower catalog manager: success
I20260806 07:49:11.332442 28775 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: acquiring CA information for follower catalog manager: success
I20260806 07:49:11.335721 28775 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:49:11.483625 28939 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:11.483968 28939 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:11.484040 28939 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:11.493096 28939 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:11.493330 28939 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:49:11.505343 28939 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:11.507093 28939 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:11.509095 28939 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:11.519104 28946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:11.519446 28947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:11.520102 28949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:11.520584 28939 server_base.cc:1061] running on GCE node
I20260806 07:49:11.521210 28939 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:11.521423 28939 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:11.521522 28939 hybrid_clock.cc:648] HybridClock initialized: now 1786002551521478 us; error 0 us; skew 500 ppm
I20260806 07:49:11.521895 28939 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:11.524115 28939 webserver.cc:533] Webserver started at http://127.20.202.194:42777/ using document root <none> and password file <none>
I20260806 07:49:11.524893 28939 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:11.525036 28939 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:11.525391 28939 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:11.528012 28939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "248e9559fa00421a859ef9930848e03b"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:11.528797 28939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "248e9559fa00421a859ef9930848e03b"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:11.534183 28939 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 07:49:11.537531 28954 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:11.538893 28939 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:11.539081 28939 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "248e9559fa00421a859ef9930848e03b"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:11.539289 28939 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:11.558112 28800 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:11.558247 28800 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: loading cluster ID for follower catalog manager: success
I20260806 07:49:11.559780 28939 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:11.561194 28939 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:11.561543 28939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:11.562821 28939 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:11.562849 28800 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: acquiring CA information for follower catalog manager: success
I20260806 07:49:11.565236 28939 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:11.565361 28939 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:11.565490 28939 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:11.565580 28939 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:11.565773 28800 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:11.618610 28939 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:43133
I20260806 07:49:11.618646 29070 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:43133 every 8 connection(s)
I20260806 07:49:11.620201 28939 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:49:11.623428 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28939
I20260806 07:49:11.623696 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:49:11.632686 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--time_source=system_unsync with env {}
I20260806 07:49:11.662345 29071 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:11.662729 29071 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:11.663857 29071 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:11.665953 28704 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:43133)
I20260806 07:49:11.667780 29072 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:11.667917 29072 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:11.668418 29072 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:11.669435 28562 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:43133)
I20260806 07:49:11.670262 28562 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:52041
I20260806 07:49:11.672473 29073 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:11.672631 29073 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:11.673265 29073 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:11.674474 28633 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:43133)
W20260806 07:49:12.052578 29079 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:12.053045 29079 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:12.053190 29079 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:12.062201 29079 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:12.062449 29079 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:49:12.072515 29079 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:12.074525 29079 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:12.076434 29079 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:12.087338 29088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:12.087498 29087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:12.088418 29090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:12.090039 29079 server_base.cc:1061] running on GCE node
I20260806 07:49:12.091168 29079 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:12.091423 29079 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:12.091537 29079 hybrid_clock.cc:648] HybridClock initialized: now 1786002552091489 us; error 0 us; skew 500 ppm
I20260806 07:49:12.091835 29079 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:12.097052 29079 webserver.cc:533] Webserver started at http://127.20.202.195:43011/ using document root <none> and password file <none>
I20260806 07:49:12.097916 29079 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:12.098042 29079 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:12.098351 29079 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:12.100807 29079 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "12d097b7637f42d4ac746c0ea859f3cd"
format_stamp: "Formatted at 2026-08-06 07:49:12 on dist-test-slave-g350"
I20260806 07:49:12.101577 29079 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "12d097b7637f42d4ac746c0ea859f3cd"
format_stamp: "Formatted at 2026-08-06 07:49:12 on dist-test-slave-g350"
I20260806 07:49:12.107743 29079 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:49:12.111413 29095 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:12.112891 29079 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:49:12.113112 29079 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "12d097b7637f42d4ac746c0ea859f3cd"
format_stamp: "Formatted at 2026-08-06 07:49:12 on dist-test-slave-g350"
I20260806 07:49:12.113327 29079 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:12.129391 29079 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:12.130352 29079 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:12.130659 29079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:12.130792 28932 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
I20260806 07:49:12.131944 29079 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:12.133898 29079 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:12.133994 29079 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:12.134085 29079 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:12.134140 29079 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:12.174104 29079 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:39951
I20260806 07:49:12.174134 29209 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:39951 every 8 connection(s)
I20260806 07:49:12.175652 29079 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:49:12.185531 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29079
I20260806 07:49:12.185757 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:49:12.207635 29210 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:12.208055 29210 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:12.208344 29211 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:12.208509 29211 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:12.209164 29211 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:12.209190 29210 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:12.211099 28562 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:39951)
I20260806 07:49:12.211148 28704 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:39951)
I20260806 07:49:12.212137 28562 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:50173
I20260806 07:49:12.216555 29212 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:12.216774 29212 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:12.217304 29212 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:12.218396 28633 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:39951)
I20260806 07:49:12.223722 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:49:12.231005 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:49:12.231168 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28530
I20260806 07:49:12.232005 28592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:12.362526 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28530
W20260806 07:49:12.381330 28689 connection.cc:573] server connection from 127.0.0.1:33412 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:49:12.383036 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--webserver_interface=127.20.202.254
--webserver_port=39709
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
I20260806 07:49:12.673440 29072 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
W20260806 07:49:12.675325 29072 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:42007 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:42007: connect: Connection refused (error 111)
W20260806 07:49:12.727386 29221 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:12.727759 29221 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:12.727909 29221 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:12.736938 29221 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:12.737063 29221 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:12.737162 29221 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:12.737264 29221 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:12.747740 29221 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=39709
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:12.749442 29221 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:12.751362 29221 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:12.760437 29228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:12.760900 29227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:12.761695 29230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:12.762120 29221 server_base.cc:1061] running on GCE node
I20260806 07:49:12.762887 29221 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:12.763032 29221 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:12.763098 29221 hybrid_clock.cc:648] HybridClock initialized: now 1786002552763080 us; error 0 us; skew 500 ppm
I20260806 07:49:12.763371 29221 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:12.766177 29221 webserver.cc:533] Webserver started at http://127.20.202.254:39709/ using document root <none> and password file <none>
I20260806 07:49:12.766917 29221 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:12.767030 29221 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:12.772569 29221 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:49:12.775556 29235 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:12.777113 29221 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:49:12.777251 29221 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "556b9231464f451f80620b7995c6b2d0"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:12.778041 29221 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:12.821532 29221 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:12.822681 29221 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:12.823005 29221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:12.843770 29221 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:42007
I20260806 07:49:12.843822 29287 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:42007 every 8 connection(s)
I20260806 07:49:12.845721 29221 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:49:12.852921 29288 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:49:12.854349 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29221
I20260806 07:49:12.854660 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28784
I20260806 07:49:12.857028 29288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap starting.
I20260806 07:49:12.864744 28926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:12.877169 29288 log.cc:826] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:12.886787 29288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:49:12.887326 29288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap complete.
I20260806 07:49:12.894117 29288 raft_consensus.cc:359] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:12.894815 29288 raft_consensus.cc:740] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556b9231464f451f80620b7995c6b2d0, State: Initialized, Role: FOLLOWER
I20260806 07:49:12.895632 29288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:12.896605 29295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:12.896915 29295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:12.897738 29288 sys_catalog.cc:565] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:12.908222 29307 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:12.908391 29307 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: loading cluster ID for follower catalog manager: success
I20260806 07:49:12.913369 29307 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: acquiring CA information for follower catalog manager: success
I20260806 07:49:12.917672 29307 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:12.947191 29221 master_runner.cc:264] No leader master found from master 556b9231464f451f80620b7995c6b2d0
I20260806 07:49:12.947365 29221 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:12.981676 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28784
I20260806 07:49:13.000806 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38255
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--webserver_port=44407
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
I20260806 07:49:13.215090 29211 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
I20260806 07:49:13.226653 29252 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "12d097b7637f42d4ac746c0ea859f3cd" instance_seqno: 1786002552164021) as {username='slave'} at 127.20.202.195:40921; Asking this server to re-register.
I20260806 07:49:13.227623 29211 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:13.229354 29252 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:39951)
W20260806 07:49:13.347622 29314 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:13.347941 29314 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:13.348017 29314 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:13.356220 29314 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:13.356375 29314 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:13.366253 29314 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38255
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44407
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:13.367826 29314 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:13.369565 29314 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:13.379264 29321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:13.379313 29320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:13.380209 29323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:13.380815 29314 server_base.cc:1061] running on GCE node
I20260806 07:49:13.381371 29314 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:13.381587 29314 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:13.381713 29314 hybrid_clock.cc:648] HybridClock initialized: now 1786002553381691 us; error 0 us; skew 500 ppm
I20260806 07:49:13.382073 29314 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:13.384423 29314 webserver.cc:533] Webserver started at http://127.20.202.193:44407/ using document root <none> and password file <none>
I20260806 07:49:13.385187 29314 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:13.385339 29314 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:13.390856 29314 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:49:13.393659 29328 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:13.395394 29314 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:49:13.395527 29314 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:13.396198 29314 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:13.415639 29314 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:13.416476 29314 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:13.416723 29314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:13.417698 29314 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:13.419472 29314 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:13.419548 29314 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:13.419638 29314 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:13.419683 29314 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:13.453182 29314 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38255
I20260806 07:49:13.453219 29441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38255 every 8 connection(s)
I20260806 07:49:13.454862 29314 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:13.463428 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29314
I20260806 07:49:13.475652 29442 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:13.476074 29442 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:13.477205 29442 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:13.477794 29445 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:13.477836 29444 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:13.478008 29445 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:13.478010 29444 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:13.478755 29445 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:13.479478 28704 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:13.478708 29444 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:13.480165 28633 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:13.480897 29252 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:13.515515 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29314
I20260806 07:49:13.526280 29437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:13.611979 29456 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 556b9231464f451f80620b7995c6b2d0)
I20260806 07:49:13.612334 29456 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:13.615233 29456 leader_election.cc:290] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 556b9231464f451f80620b7995c6b2d0 (127.20.202.254:42007), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:13.615280 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29314
I20260806 07:49:13.618079 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa" is_pre_election: true
I20260806 07:49:13.625025 29262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "556b9231464f451f80620b7995c6b2d0" is_pre_election: true
I20260806 07:49:13.625552 29262 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a09d3451a5c4d0aa118905164318b9b in term 1.
I20260806 07:49:13.626181 28617 leader_election.cc:304] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b; no voters: 28bee9a34538401481754941e47058fa
I20260806 07:49:13.626602 29456 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:49:13.626744 29456 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 556b9231464f451f80620b7995c6b2d0)
I20260806 07:49:13.626865 29456 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:49:13.628996 29456 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:13.629634 29456 leader_election.cc:290] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 2 election: Requested vote from peers 556b9231464f451f80620b7995c6b2d0 (127.20.202.254:42007), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:13.630193 29262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "556b9231464f451f80620b7995c6b2d0"
I20260806 07:49:13.630268 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa"
I20260806 07:49:13.630402 29262 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:49:13.634428 29262 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a09d3451a5c4d0aa118905164318b9b in term 2.
I20260806 07:49:13.634991 28617 leader_election.cc:304] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b; no voters: 28bee9a34538401481754941e47058fa
I20260806 07:49:13.635388 29456 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:49:13.635877 29456 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 LEADER]: Becoming Leader. State: Replica: 6a09d3451a5c4d0aa118905164318b9b, State: Running, Role: LEADER
I20260806 07:49:13.636483 29456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:13.637214 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38255
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--webserver_port=44407
--webserver_interface=127.20.202.193
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260806 07:49:13.638374 29458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a09d3451a5c4d0aa118905164318b9b. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:13.638659 29458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:13.639415 29461 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:49:13.642318 29461 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:49:13.643456 29461 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:13.643551 29461 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:49:13.645094 29461 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:49:13.646083 29461 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Loaded TSK: 0
I20260806 07:49:13.647228 29461 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:49:13.683743 29072 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:13.683965 29072 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
I20260806 07:49:13.685002 29252 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "248e9559fa00421a859ef9930848e03b" instance_seqno: 1786002551603024) as {username='slave'} at 127.20.202.194:60659; Asking this server to re-register.
I20260806 07:49:13.685639 29072 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:13.686746 29252 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:43133)
I20260806 07:49:13.864187 29464 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 556b9231464f451f80620b7995c6b2d0)
I20260806 07:49:13.864449 29464 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:13.865968 29464 leader_election.cc:290] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 556b9231464f451f80620b7995c6b2d0 (127.20.202.254:42007), 6a09d3451a5c4d0aa118905164318b9b (127.20.202.253:34181)
I20260806 07:49:13.867193 28642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28bee9a34538401481754941e47058fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a09d3451a5c4d0aa118905164318b9b" is_pre_election: true
I20260806 07:49:13.873148 29262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28bee9a34538401481754941e47058fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "556b9231464f451f80620b7995c6b2d0" is_pre_election: true
I20260806 07:49:13.873512 29262 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bee9a34538401481754941e47058fa in current term 2: Already voted for candidate 6a09d3451a5c4d0aa118905164318b9b in this term.
I20260806 07:49:13.874269 28688 leader_election.cc:304] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bee9a34538401481754941e47058fa; no voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b
I20260806 07:49:13.874697 29464 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:49:13.878041 29464 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 07:49:13.990567 29462 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:13.990859 29462 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:13.990933 29462 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:49:13.991003 29462 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:13.999251 29462 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:13.999477 29462 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:14.009289 29462 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38255
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44407
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:14.011031 29462 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:14.012813 29462 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:14.022012 29470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:14.022761 29471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:14.023326 29473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:14.023852 29462 server_base.cc:1061] running on GCE node
I20260806 07:49:14.024416 29462 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:14.024591 29462 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:14.024709 29462 hybrid_clock.cc:648] HybridClock initialized: now 1786002554024664 us; error 0 us; skew 500 ppm
I20260806 07:49:14.025074 29462 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:14.027333 29462 webserver.cc:533] Webserver started at http://127.20.202.193:44407/ using document root <none> and password file <none>
I20260806 07:49:14.028023 29462 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:14.028137 29462 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:14.033583 29462 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:14.036634 29478 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:14.038153 29462 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:14.038316 29462 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:14.039067 29462 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:14.067029 28715 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 2 FOLLOWER]: Refusing update from remote peer 6a09d3451a5c4d0aa118905164318b9b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:49:14.067214 29462 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:14.067970 29458 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:49:14.068404 29462 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:14.068754 29462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:14.069913 29462 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:14.070188 29464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a09d3451a5c4d0aa118905164318b9b. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:14.070437 29464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:14.072158 29462 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:14.072278 29462 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:14.072381 29462 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:14.072468 29462 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:49:14.072906 29462 fault_injection.cc:43] FAULT INJECTION ENABLED!
I20260806 07:49:14.072796 29456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
W20260806 07:49:14.073024 29462 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20260806 07:49:14.073087 29456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: LEADER
E20260806 07:49:14.073103 29462 fault_injection.cc:59] injecting fault for kudu (pid 29462): FLAGS_fault_before_start (process will exit)
I20260806 07:49:14.074210 29464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:14.074465 29464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:14.075245 29262 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Refusing update from remote peer 6a09d3451a5c4d0aa118905164318b9b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:49:14.076032 29458 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:49:14.084524 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:14.084908 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:14.085125 29489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a09d3451a5c4d0aa118905164318b9b. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:14.085332 29489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:14.091794 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38255
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--webserver_port=44407
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
W20260806 07:49:14.433777 29493 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:14.434131 29493 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:14.434270 29493 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:14.442624 29493 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:14.442838 29493 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:14.452502 29493 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38255
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44407
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:14.454211 29493 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:14.456094 29493 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:14.465823 29498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:14.465965 29499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:14.466483 29501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:14.467650 29493 server_base.cc:1061] running on GCE node
I20260806 07:49:14.468273 29493 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:14.468549 29493 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:14.468659 29493 hybrid_clock.cc:648] HybridClock initialized: now 1786002554468638 us; error 0 us; skew 500 ppm
I20260806 07:49:14.468928 29493 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:14.471187 29493 webserver.cc:533] Webserver started at http://127.20.202.193:44407/ using document root <none> and password file <none>
I20260806 07:49:14.471935 29493 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:14.472044 29493 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:14.477577 29493 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:14.480211 29506 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:14.481762 29493 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:49:14.481936 29493 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:14.482649 29493 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:14.501425 29493 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:14.502292 29493 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:14.502538 29493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:14.503403 29493 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:14.505026 29493 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:14.505100 29493 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:14.505182 29493 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:14.505226 29493 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:14.540416 29493 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38255
I20260806 07:49:14.540450 29620 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38255 every 8 connection(s)
I20260806 07:49:14.542273 29493 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:14.544248 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29493
I20260806 07:49:14.544570 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29493
I20260806 07:49:14.576421 29621 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:14.576861 29621 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:14.576902 29622 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:14.577065 29622 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:14.578269 29622 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:14.578293 29621 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:14.579013 29623 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:14.579177 29623 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:14.579778 29623 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:14.580410 28704 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:14.580710 29252 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:14.580788 28633 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:38255)
I20260806 07:49:14.582247 28633 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:51073
I20260806 07:49:14.583971 29616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:14.673780 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29493
I20260806 07:49:14.679306 29073 heartbeater.cc:499] Master 127.20.202.253:34181 was elected leader, sending a full tablet report...
I20260806 07:49:14.691067 29634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:14.691583 29634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:14.693833 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28939
I20260806 07:49:14.702203 29066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:14.788090 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28939
I20260806 07:49:14.805833 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29079
I20260806 07:49:14.814683 29205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:14.908571 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29079
I20260806 07:49:14.927094 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29221
I20260806 07:49:14.933296 29282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:15.039393 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29221
I20260806 07:49:15.061698 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28601
I20260806 07:49:15.062862 28663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:15.175932 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28601
I20260806 07:49:15.198390 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28670
I20260806 07:49:15.199551 28735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:15.305764 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28670
Shutting down hive metastore.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:49:16: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:49:24.820808 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--webserver_interface=127.20.202.254
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
W20260806 07:49:25.163177 29830 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:25.163511 29830 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:25.163578 29830 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:25.172003 29830 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:25.172094 29830 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:25.172152 29830 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:25.172196 29830 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:25.182792 29830 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:25.185305 29830 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:25.187839 29830 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:25.196282 29835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:25.196331 29836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:25.197847 29830 server_base.cc:1061] running on GCE node
W20260806 07:49:25.198715 29838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:25.199581 29830 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:25.199770 29830 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:25.199882 29830 hybrid_clock.cc:648] HybridClock initialized: now 1786002565199839 us; error 0 us; skew 500 ppm
I20260806 07:49:25.200248 29830 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:25.203050 29830 webserver.cc:533] Webserver started at http://127.20.202.254:39913/ using document root <none> and password file <none>
I20260806 07:49:25.203817 29830 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:25.203963 29830 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:25.209410 29830 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:49:25.213423 29843 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:25.214819 29830 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:49:25.215036 29830 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "556b9231464f451f80620b7995c6b2d0"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:25.215752 29830 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:25.235759 29830 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:25.236660 29830 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:25.236972 29830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:25.258689 29830 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:42007
I20260806 07:49:25.258749 29895 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:42007 every 8 connection(s)
I20260806 07:49:25.260455 29830 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:49:25.263876 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29830
I20260806 07:49:25.264094 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:49:25.265317 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34181
--webserver_interface=127.20.202.253
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
I20260806 07:49:25.269919 29896 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:49:25.274633 29896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap starting.
I20260806 07:49:25.291514 29896 log.cc:826] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:25.300498 29896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:49:25.301045 29896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap complete.
I20260806 07:49:25.307789 29896 raft_consensus.cc:359] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:25.308491 29896 raft_consensus.cc:740] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556b9231464f451f80620b7995c6b2d0, State: Initialized, Role: FOLLOWER
I20260806 07:49:25.309243 29896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:25.310187 29900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:25.310503 29900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:25.311199 29896 sys_catalog.cc:565] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:25.321722 29912 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:25.321871 29912 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: loading cluster ID for follower catalog manager: success
I20260806 07:49:25.326155 29912 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: acquiring CA information for follower catalog manager: success
I20260806 07:49:25.328778 29912 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:25.341457 29830 master_runner.cc:187] Error getting master registration for 127.20.202.253:34181: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34181: connect: Connection refused (error 111),
I20260806 07:49:25.345223 29830 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:49:25.635123 29898 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:25.635514 29898 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:25.635731 29898 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:25.644862 29898 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:25.645002 29898 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:25.645107 29898 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:25.645195 29898 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:25.655766 29898 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:25.657557 29898 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:25.659629 29898 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:25.670099 29920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:25.670037 29921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:25.671947 29923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:25.672803 29898 server_base.cc:1061] running on GCE node
I20260806 07:49:25.673579 29898 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:25.673861 29898 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:25.673978 29898 hybrid_clock.cc:648] HybridClock initialized: now 1786002565673949 us; error 0 us; skew 500 ppm
I20260806 07:49:25.674340 29898 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:25.677138 29898 webserver.cc:533] Webserver started at http://127.20.202.253:34655/ using document root <none> and password file <none>
I20260806 07:49:25.678035 29898 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:25.678238 29898 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:25.683976 29898 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:49:25.687055 29928 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:25.688594 29898 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:25.688781 29898 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "6a09d3451a5c4d0aa118905164318b9b"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:25.689554 29898 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:25.704483 29898 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:25.705446 29898 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:25.705813 29898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:25.728199 29898 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:34181
I20260806 07:49:25.728252 29979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:34181 every 8 connection(s)
I20260806 07:49:25.730053 29898 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:49:25.737375 29980 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:49:25.739691 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29898
I20260806 07:49:25.739928 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:49:25.740926 29980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Bootstrap starting.
I20260806 07:49:25.741113 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:43789
--webserver_interface=127.20.202.252
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
I20260806 07:49:25.760795 29980 log.cc:826] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:25.773435 29980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:49:25.774108 29980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Bootstrap complete.
I20260806 07:49:25.783963 29980 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:25.784869 29980 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a09d3451a5c4d0aa118905164318b9b, State: Initialized, Role: FOLLOWER
I20260806 07:49:25.785704 29980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:25.786593 29984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:25.786878 29984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:25.787853 29980 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:25.802532 29995 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:25.802717 29995 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: loading cluster ID for follower catalog manager: success
I20260806 07:49:25.806859 29995 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: acquiring CA information for follower catalog manager: success
I20260806 07:49:25.809580 29995 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:25.830672 29898 master_runner.cc:187] Error getting master registration for 127.20.202.252:43789: Network error: Client connection negotiation failed: client connection to 127.20.202.252:43789: connect: Connection refused (error 111),
I20260806 07:49:25.836362 29898 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:49:26.121110 29982 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:26.121513 29982 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:26.121671 29982 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:26.130254 29982 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:26.130347 29982 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:26.130436 29982 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:26.130533 29982 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:26.142716 29982 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:43789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:26.144466 29982 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:26.146492 29982 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:26.154924 30003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:26.155781 30004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:26.155844 30006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:26.157437 29982 server_base.cc:1061] running on GCE node
I20260806 07:49:26.158278 29982 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:26.158479 29982 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:26.158589 29982 hybrid_clock.cc:648] HybridClock initialized: now 1786002566158565 us; error 0 us; skew 500 ppm
I20260806 07:49:26.158939 29982 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:26.161976 29982 webserver.cc:533] Webserver started at http://127.20.202.252:46649/ using document root <none> and password file <none>
I20260806 07:49:26.162822 29982 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:26.162992 29982 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:26.169106 29982 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:49:26.172341 30011 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:26.173902 29982 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:26.174113 29982 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "28bee9a34538401481754941e47058fa"
format_stamp: "Formatted at 2026-08-06 07:49:10 on dist-test-slave-g350"
I20260806 07:49:26.174846 29982 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:26.193861 29982 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:26.194902 29982 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:26.195214 29982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:26.216764 29982 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:43789
I20260806 07:49:26.216800 30064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:43789 every 8 connection(s)
I20260806 07:49:26.218672 29982 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:49:26.219920 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29982
I20260806 07:49:26.220109 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:49:26.228274 30065 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:49:26.232379 30065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Bootstrap starting.
I20260806 07:49:26.250504 30065 log.cc:826] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:26.259586 30065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:49:26.260190 30065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: Bootstrap complete.
I20260806 07:49:26.266923 30065 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:26.267647 30065 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bee9a34538401481754941e47058fa, State: Initialized, Role: FOLLOWER
I20260806 07:49:26.268365 30065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:26.269307 30070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:26.269690 30070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:26.270498 30065 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:26.282230 30081 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:26.282336 30081 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: loading cluster ID for follower catalog manager: success
I20260806 07:49:26.287415 30081 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: acquiring CA information for follower catalog manager: success
I20260806 07:49:26.290884 30081 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:26.305953 29982 master_runner.cc:264] No leader master found from master 28bee9a34538401481754941e47058fa
I20260806 07:49:26.306128 29982 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:26.627184 30085 raft_consensus.cc:493] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:49:26.627480 30085 raft_consensus.cc:515] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:26.629531 30085 leader_election.cc:290] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a09d3451a5c4d0aa118905164318b9b (127.20.202.253:34181), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:26.637935 29955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a09d3451a5c4d0aa118905164318b9b" is_pre_election: true
I20260806 07:49:26.638053 30039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa" is_pre_election: true
I20260806 07:49:26.638530 29955 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 556b9231464f451f80620b7995c6b2d0 in term 2.
I20260806 07:49:26.638619 30039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 556b9231464f451f80620b7995c6b2d0 in term 2.
I20260806 07:49:26.639273 29844 leader_election.cc:304] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b; no voters:
I20260806 07:49:26.639768 30085 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:49:26.639931 30085 raft_consensus.cc:493] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:49:26.640036 30085 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:49:26.644359 30085 raft_consensus.cc:515] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:26.645128 30085 leader_election.cc:290] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 3 election: Requested vote from peers 6a09d3451a5c4d0aa118905164318b9b (127.20.202.253:34181), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:26.645735 29955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a09d3451a5c4d0aa118905164318b9b"
I20260806 07:49:26.645773 30039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "556b9231464f451f80620b7995c6b2d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa"
I20260806 07:49:26.645987 30039 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:49:26.645983 29955 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:49:26.651580 29955 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 556b9231464f451f80620b7995c6b2d0 in term 3.
I20260806 07:49:26.652159 29844 leader_election.cc:304] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b; no voters:
I20260806 07:49:26.652253 30039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 556b9231464f451f80620b7995c6b2d0 in term 3.
I20260806 07:49:26.652585 30085 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:49:26.653369 30085 raft_consensus.cc:697] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 LEADER]: Becoming Leader. State: Replica: 556b9231464f451f80620b7995c6b2d0, State: Running, Role: LEADER
I20260806 07:49:26.653993 30085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:26.657094 30088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556b9231464f451f80620b7995c6b2d0. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:26.657534 30088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:26.658391 30090 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:49:26.660547 30090 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:49:26.661439 30090 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:26.661556 30090 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:49:26.662716 30090 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:49:26.663512 30090 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Loaded TSK: 0
I20260806 07:49:26.664395 30090 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:49:26.685894 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--time_source=system_unsync with env {}
W20260806 07:49:27.037786 30091 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:27.038159 30091 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:27.038247 30091 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:27.046361 30091 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:27.046514 30091 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:27.056492 30091 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:27.058178 30091 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:27.060052 30091 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:27.070675 30096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:27.072091 30097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:27.072144 30099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:27.072953 30091 server_base.cc:1061] running on GCE node
I20260806 07:49:27.073530 30091 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:27.073746 30091 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:27.073846 30091 hybrid_clock.cc:648] HybridClock initialized: now 1786002567073824 us; error 0 us; skew 500 ppm
I20260806 07:49:27.074139 30091 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:27.076496 30091 webserver.cc:533] Webserver started at http://127.20.202.193:41233/ using document root <none> and password file <none>
I20260806 07:49:27.077246 30091 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:27.077344 30091 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:27.080796 29955 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Refusing update from remote peer 556b9231464f451f80620b7995c6b2d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:49:27.081880 30088 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:49:27.083743 30091 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 07:49:27.085811 30104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556b9231464f451f80620b7995c6b2d0. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:27.086233 30104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:27.087150 30085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:27.087373 30085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:27.088102 30107 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:27.088179 30039 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 3 FOLLOWER]: Refusing update from remote peer 556b9231464f451f80620b7995c6b2d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:49:27.089016 30088 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:49:27.089349 30104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:27.089557 30104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:27.090621 30091 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:49:27.090806 30091 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:27.091571 30091 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:27.094410 30112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:27.094687 30112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:27.095418 30109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556b9231464f451f80620b7995c6b2d0. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:27.095709 30109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:27.107750 30091 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:27.108722 30091 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:27.109014 30091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:27.110002 30091 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:27.111670 30091 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:27.111773 30091 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:27.111915 30091 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:27.111991 30091 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:27.148531 30091 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41281
I20260806 07:49:27.148576 30226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41281 every 8 connection(s)
I20260806 07:49:27.150187 30091 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:27.159631 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30091
I20260806 07:49:27.159879 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:49:27.167290 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--time_source=system_unsync with env {}
I20260806 07:49:27.182209 30227 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:27.182628 30227 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:27.183426 30229 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:27.183643 30229 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:27.183723 30227 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:27.184305 30229 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:27.186509 30028 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:41281)
I20260806 07:49:27.186507 29945 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:41281)
I20260806 07:49:27.189939 30228 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:27.190092 30228 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:27.190565 30228 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:27.192055 29859 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:41281)
I20260806 07:49:27.193681 29859 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:53867
I20260806 07:49:27.478955 30085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "556b9231464f451f80620b7995c6b2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:27.479311 30085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: LEADER
W20260806 07:49:27.518077 30237 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:27.518414 30237 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:27.518563 30237 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:27.526791 30237 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:27.526995 30237 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:49:27.536816 30237 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:27.538532 30237 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:27.540360 30237 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:27.550532 30243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:27.550830 30244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:27.551375 30246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:27.556730 30237 server_base.cc:1061] running on GCE node
I20260806 07:49:27.557355 30237 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:27.557556 30237 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:27.557691 30237 hybrid_clock.cc:648] HybridClock initialized: now 1786002567557671 us; error 0 us; skew 500 ppm
I20260806 07:49:27.557982 30237 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:27.560192 30237 webserver.cc:533] Webserver started at http://127.20.202.194:41605/ using document root <none> and password file <none>
I20260806 07:49:27.560959 30237 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:27.561079 30237 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:27.566746 30237 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:49:27.569437 30251 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:27.570852 30237 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:27.571041 30237 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "248e9559fa00421a859ef9930848e03b"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:27.571786 30237 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:27.588430 30237 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:27.589296 30237 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:27.589643 30237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:27.590615 30237 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:27.592223 30237 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:27.592295 30237 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:27.592379 30237 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:27.592423 30237 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:27.626974 30237 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:39417
I20260806 07:49:27.627027 30364 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:39417 every 8 connection(s)
I20260806 07:49:27.628520 30237 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:49:27.632494 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30237
I20260806 07:49:27.632682 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:49:27.638141 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--time_source=system_unsync with env {}
I20260806 07:49:27.657648 30366 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:27.657675 30365 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:27.658146 30366 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:27.658152 30365 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:27.658411 30367 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:27.658547 30367 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:27.659497 30367 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:27.659457 30365 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:27.659587 30366 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:27.661482 29859 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:39417)
I20260806 07:49:27.661540 30028 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:39417)
I20260806 07:49:27.661563 29945 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:39417)
I20260806 07:49:27.662628 29859 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:57749
W20260806 07:49:27.981818 30372 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:27.982198 30372 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:27.982332 30372 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:27.990792 30372 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:27.991007 30372 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:49:28.001144 30372 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:28.002949 30372 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:28.004824 30372 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:28.014750 30378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:28.014449 30377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:28.015316 30380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:28.016412 30372 server_base.cc:1061] running on GCE node
I20260806 07:49:28.016975 30372 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:28.017143 30372 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:28.017283 30372 hybrid_clock.cc:648] HybridClock initialized: now 1786002568017257 us; error 0 us; skew 500 ppm
I20260806 07:49:28.017673 30372 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:28.019873 30372 webserver.cc:533] Webserver started at http://127.20.202.195:36835/ using document root <none> and password file <none>
I20260806 07:49:28.020649 30372 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:28.020800 30372 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:28.026692 30372 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:49:28.029357 30386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:28.030716 30372 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:28.030892 30372 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "12d097b7637f42d4ac746c0ea859f3cd"
format_stamp: "Formatted at 2026-08-06 07:49:12 on dist-test-slave-g350"
I20260806 07:49:28.031625 30372 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:28.049997 30372 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:28.050844 30372 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:28.051141 30372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:28.052063 30372 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:28.053802 30372 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:28.053905 30372 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:28.054054 30372 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:28.054128 30372 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:28.094296 30372 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:43095
I20260806 07:49:28.094305 30500 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:43095 every 8 connection(s)
I20260806 07:49:28.096108 30372 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:49:28.102640 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30372
I20260806 07:49:28.102864 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:49:28.115113 30501 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:28.115336 30502 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:28.115677 30501 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:28.115684 30502 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:28.116412 30503 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:28.116631 30503 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:28.117080 30502 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:28.117120 30501 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:28.117321 30503 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:28.119288 30028 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:43095)
I20260806 07:49:28.119336 29859 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:43095)
I20260806 07:49:28.119688 29945 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:43095)
I20260806 07:49:28.120682 29859 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:44155
I20260806 07:49:28.128315 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:49:28.135288 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:49:28.135430 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29830
I20260806 07:49:28.136198 29891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:28.196856 30228 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
I20260806 07:49:28.256060 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29830
I20260806 07:49:28.280391 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--webserver_interface=127.20.202.254
--webserver_port=39913
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789 with env {}
W20260806 07:49:28.622285 30509 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:28.622684 30509 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:28.622834 30509 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:28.634235 30509 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:28.634315 30509 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:28.634413 30509 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:28.634491 30509 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:28.644251 30509 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=39913
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:28.645830 30509 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:28.647990 30509 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:28.657095 30514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:28.657119 30515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:28.657418 30517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:28.658913 30509 server_base.cc:1061] running on GCE node
I20260806 07:49:28.659605 30509 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:28.659749 30509 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:28.659813 30509 hybrid_clock.cc:648] HybridClock initialized: now 1786002568659795 us; error 0 us; skew 500 ppm
I20260806 07:49:28.660117 30509 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:28.662817 30509 webserver.cc:533] Webserver started at http://127.20.202.254:39913/ using document root <none> and password file <none>
I20260806 07:49:28.663830 30509 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:28.663961 30509 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:28.665261 30366 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
W20260806 07:49:28.666690 30366 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:42007 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:42007: connect: Connection refused (error 111)
I20260806 07:49:28.670015 30509 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:49:28.673151 30524 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:28.674588 30509 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:28.674767 30509 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "556b9231464f451f80620b7995c6b2d0"
format_stamp: "Formatted at 2026-08-06 07:49:09 on dist-test-slave-g350"
I20260806 07:49:28.675527 30509 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:28.694239 30509 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:28.695175 30509 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:28.695473 30509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:28.716107 30509 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:42007
I20260806 07:49:28.716151 30576 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:42007 every 8 connection(s)
I20260806 07:49:28.717765 30509 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:49:28.722128 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30509
I20260806 07:49:28.722437 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30091
I20260806 07:49:28.725585 30577 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:49:28.729863 30577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap starting.
I20260806 07:49:28.733099 30222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:28.748131 30577 log.cc:826] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:28.758038 30577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:49:28.758641 30577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: Bootstrap complete.
I20260806 07:49:28.765429 30577 raft_consensus.cc:359] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:28.766299 30577 raft_consensus.cc:740] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556b9231464f451f80620b7995c6b2d0, State: Initialized, Role: FOLLOWER
I20260806 07:49:28.767139 30577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:28.768142 30583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:28.768473 30583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:28.769090 30577 sys_catalog.cc:565] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:28.779486 30594 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:28.779731 30594 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: loading cluster ID for follower catalog manager: success
I20260806 07:49:28.784183 30594 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: acquiring CA information for follower catalog manager: success
I20260806 07:49:28.786892 30594 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:28.831063 30509 master_runner.cc:264] No leader master found from master 556b9231464f451f80620b7995c6b2d0
I20260806 07:49:28.831240 30509 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:28.838025 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30091
I20260806 07:49:28.856873 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41281
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--webserver_port=41233
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
I20260806 07:49:29.123023 30502 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
I20260806 07:49:29.134187 30541 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "12d097b7637f42d4ac746c0ea859f3cd" instance_seqno: 1786002568084910) as {username='slave'} at 127.20.202.195:56153; Asking this server to re-register.
I20260806 07:49:29.135144 30502 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:29.137059 30541 ts_manager.cc:194] Registered new tserver with Master: 12d097b7637f42d4ac746c0ea859f3cd (127.20.202.195:43095)
W20260806 07:49:29.207002 30600 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:29.207342 30600 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:29.207486 30600 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:29.215911 30600 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:29.216125 30600 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:29.226161 30600 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41281
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41233
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:29.227768 30600 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:29.229570 30600 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:29.239718 30606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:29.240449 30607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:29.240502 30609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:29.241223 30600 server_base.cc:1061] running on GCE node
I20260806 07:49:29.241885 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:29.242067 30600 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:29.242154 30600 hybrid_clock.cc:648] HybridClock initialized: now 1786002569242136 us; error 0 us; skew 500 ppm
I20260806 07:49:29.242451 30600 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:29.244671 30600 webserver.cc:533] Webserver started at http://127.20.202.193:41233/ using document root <none> and password file <none>
I20260806 07:49:29.245450 30600 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:29.245663 30600 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:29.250967 30600 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:49:29.253559 30614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:29.254915 30600 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:49:29.255086 30600 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:29.255842 30600 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:29.280897 30600 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:29.281919 30600 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:29.282218 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:29.283214 30600 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:29.284938 30600 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:29.285043 30600 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:29.285204 30600 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:29.285279 30600 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:29.321130 30600 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41281
I20260806 07:49:29.321187 30729 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41281 every 8 connection(s)
I20260806 07:49:29.322914 30600 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:29.329875 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30600
I20260806 07:49:29.342516 30730 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:29.342989 30730 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:29.343343 30731 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:29.343569 30731 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:29.344434 30731 heartbeater.cc:507] Master 127.20.202.254:42007 requested a full tablet report, sending...
I20260806 07:49:29.345887 30730 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:29.346565 30541 ts_manager.cc:194] Registered new tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:41281)
I20260806 07:49:29.347328 30028 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:41281)
I20260806 07:49:29.348359 30732 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:29.348505 30732 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:29.348968 30732 heartbeater.cc:507] Master 127.20.202.253:34181 requested a full tablet report, sending...
I20260806 07:49:29.350234 29945 ts_manager.cc:194] Re-registered known tserver with Master: b1f45e00d93d41a69b01fc134c1c5007 (127.20.202.193:41281)
I20260806 07:49:29.449081 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30600
I20260806 07:49:29.458487 30725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:29.511824 30742 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 556b9231464f451f80620b7995c6b2d0)
I20260806 07:49:29.512116 30742 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:29.514151 30742 leader_election.cc:290] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 556b9231464f451f80620b7995c6b2d0 (127.20.202.254:42007), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:29.522883 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "556b9231464f451f80620b7995c6b2d0" is_pre_election: true
I20260806 07:49:29.523299 30039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa" is_pre_election: true
I20260806 07:49:29.523526 30551 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a09d3451a5c4d0aa118905164318b9b in term 3.
I20260806 07:49:29.524148 29929 leader_election.cc:304] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b; no voters: 28bee9a34538401481754941e47058fa
I20260806 07:49:29.524560 30742 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 07:49:29.524706 30742 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 556b9231464f451f80620b7995c6b2d0)
I20260806 07:49:29.524829 30742 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:49:29.525913 30745 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 556b9231464f451f80620b7995c6b2d0)
I20260806 07:49:29.526111 30745 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:29.527884 30742 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:29.528331 30745 leader_election.cc:290] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 556b9231464f451f80620b7995c6b2d0 (127.20.202.254:42007), 6a09d3451a5c4d0aa118905164318b9b (127.20.202.253:34181)
I20260806 07:49:29.529250 30742 leader_election.cc:290] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 4 election: Requested vote from peers 556b9231464f451f80620b7995c6b2d0 (127.20.202.254:42007), 28bee9a34538401481754941e47058fa (127.20.202.252:43789)
I20260806 07:49:29.529206 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "556b9231464f451f80620b7995c6b2d0"
I20260806 07:49:29.529078 30039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a09d3451a5c4d0aa118905164318b9b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bee9a34538401481754941e47058fa"
I20260806 07:49:29.529415 30551 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:49:29.529619 30039 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:49:29.532768 30039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a09d3451a5c4d0aa118905164318b9b in term 4.
I20260806 07:49:29.533254 29932 leader_election.cc:304] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bee9a34538401481754941e47058fa, 6a09d3451a5c4d0aa118905164318b9b; no voters:
I20260806 07:49:29.533663 30742 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 4 FOLLOWER]: Leader election won for term 4
I20260806 07:49:29.535663 30551 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a09d3451a5c4d0aa118905164318b9b in term 4.
I20260806 07:49:29.537462 29955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28bee9a34538401481754941e47058fa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a09d3451a5c4d0aa118905164318b9b" is_pre_election: true
I20260806 07:49:29.539434 30742 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [term 4 LEADER]: Becoming Leader. State: Replica: 6a09d3451a5c4d0aa118905164318b9b, State: Running, Role: LEADER
I20260806 07:49:29.539921 30742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } }
I20260806 07:49:29.542420 30747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a09d3451a5c4d0aa118905164318b9b. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:29.542788 30747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:29.544519 30749 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:49:29.545218 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28bee9a34538401481754941e47058fa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "556b9231464f451f80620b7995c6b2d0" is_pre_election: true
I20260806 07:49:29.545554 30551 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bee9a34538401481754941e47058fa in current term 4: Already voted for candidate 6a09d3451a5c4d0aa118905164318b9b in this term.
I20260806 07:49:29.546285 30012 leader_election.cc:304] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bee9a34538401481754941e47058fa; no voters: 556b9231464f451f80620b7995c6b2d0, 6a09d3451a5c4d0aa118905164318b9b
I20260806 07:49:29.546674 30745 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 07:49:29.547708 30749 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:49:29.548725 30749 catalog_manager.cc:1295] Loaded cluster ID: a3ab8be9c01e46648cfba369d5e2f9a3
I20260806 07:49:29.548811 30749 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:49:29.550298 30749 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:49:29.551316 30749 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b: Loaded TSK: 0
I20260806 07:49:29.552481 30749 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:49:29.562006 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30600
I20260806 07:49:29.584136 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41281
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--webserver_port=41233
--webserver_interface=127.20.202.193
--time_source=system_unsync
--fault_before_start=1.0 with env {}
I20260806 07:49:29.673864 30366 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42007
I20260806 07:49:29.674095 30366 heartbeater.cc:499] Master 127.20.202.254:42007 was elected leader, sending a full tablet report...
I20260806 07:49:29.675109 30541 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "248e9559fa00421a859ef9930848e03b" instance_seqno: 1786002567618855) as {username='slave'} at 127.20.202.194:38255; Asking this server to re-register.
I20260806 07:49:29.675763 30366 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:29.676927 30541 ts_manager.cc:194] Registered new tserver with Master: 248e9559fa00421a859ef9930848e03b (127.20.202.194:39417)
W20260806 07:49:29.928081 30750 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:29.928431 30750 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:29.928588 30750 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:49:29.928676 30750 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:29.936895 30750 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:29.937125 30750 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:29.951594 30750 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41281
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41233
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:29.953213 30750 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:29.955087 30750 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:49:29.964241 30551 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [term 4 FOLLOWER]: Refusing update from remote peer 6a09d3451a5c4d0aa118905164318b9b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:49:29.965519 30747 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 07:49:29.965731 30757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:29.965976 30756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:29.967743 30759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:29.968770 30750 server_base.cc:1061] running on GCE node
I20260806 07:49:29.969503 30750 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:29.969758 30750 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:29.969866 30750 hybrid_clock.cc:648] HybridClock initialized: now 1786002569969824 us; error 0 us; skew 500 ppm
I20260806 07:49:29.969852 30761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a09d3451a5c4d0aa118905164318b9b. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:29.970144 30761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:29.970286 30750 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:29.970988 30747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:29.971258 30747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:29.971673 30039 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [term 4 FOLLOWER]: Refusing update from remote peer 6a09d3451a5c4d0aa118905164318b9b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:49:29.972402 30742 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:49:29.973822 30761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:29.974071 30761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556b9231464f451f80620b7995c6b2d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:29.974517 30750 webserver.cc:533] Webserver started at http://127.20.202.193:41233/ using document root <none> and password file <none>
I20260806 07:49:29.975266 30750 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:29.975415 30750 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:29.980083 30770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:29.980289 30745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a09d3451a5c4d0aa118905164318b9b. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:29.980537 30770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:29.980619 30745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28bee9a34538401481754941e47058fa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:29.982422 30750 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.004s sys 0.002s
I20260806 07:49:29.985136 30772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:29.986579 30750 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:29.986758 30750 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:29.987583 30750 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:30.003147 30750 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:30.004101 30750 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:30.004400 30750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:30.005438 30750 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:30.007223 30750 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:30.007350 30750 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:30.007493 30750 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:30.007589 30750 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:49:30.008018 30750 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:49:30.008144 30750 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:49:30.008224 30750 fault_injection.cc:59] injecting fault for kudu (pid 30750): FLAGS_fault_before_start (process will exit)
I20260806 07:49:30.030124 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41281
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--webserver_port=41233
--webserver_interface=127.20.202.193
--time_source=system_unsync with env {}
W20260806 07:49:30.370903 30781 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:30.371244 30781 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:30.371348 30781 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:30.380009 30781 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:30.380193 30781 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:30.390611 30781 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41281
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41233
--tserver_master_addrs=127.20.202.254:42007,127.20.202.253:34181,127.20.202.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:30.392333 30781 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:30.394172 30781 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:30.403645 30786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:30.403738 30787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:30.404819 30789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:30.405274 30781 server_base.cc:1061] running on GCE node
I20260806 07:49:30.406010 30781 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:49:30.406201 30781 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:49:30.406299 30781 hybrid_clock.cc:648] HybridClock initialized: now 1786002570406276 us; error 0 us; skew 500 ppm
I20260806 07:49:30.406602 30781 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:30.408859 30781 webserver.cc:533] Webserver started at http://127.20.202.193:41233/ using document root <none> and password file <none>
I20260806 07:49:30.409706 30781 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:30.409852 30781 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:30.415165 30781 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:49:30.417903 30794 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:30.419247 30781 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:30.419445 30781 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "b1f45e00d93d41a69b01fc134c1c5007"
format_stamp: "Formatted at 2026-08-06 07:49:11 on dist-test-slave-g350"
I20260806 07:49:30.420161 30781 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:30.442613 30764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "6a09d3451a5c4d0aa118905164318b9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "556b9231464f451f80620b7995c6b2d0" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42007 } } peers { permanent_uuid: "6a09d3451a5c4d0aa118905164318b9b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34181 } } peers { permanent_uuid: "28bee9a34538401481754941e47058fa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 43789 } } }
I20260806 07:49:30.442905 30764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a09d3451a5c4d0aa118905164318b9b [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:30.446760 30781 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:30.447610 30781 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:30.447861 30781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:30.448805 30781 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:30.450497 30781 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:30.450572 30781 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:30.450742 30781 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:30.450795 30781 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:30.487712 30781 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41281
I20260806 07:49:30.487754 30908 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41281 every 8 connection(s)
I20260806 07:49:30.489372 30781 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:30.492167 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30781
I20260806 07:49:30.492503 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30781
I20260806 07:49:30.522053 30909 heartbeater.cc:344] Connected to a master server at 127.20.202.252:43789
I20260806 07:49:30.522298 30904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:30.522475 30909 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:30.607612 30911 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34181
I20260806 07:49:30.607990 30911 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:30.608340 30909 heartbeater.cc:507] Master 127.20.202.252:43789 requested a full tablet report, sending...
I20260806 07:49:30.609537 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30781
I20260806 07:49:30.627830 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30237
W20260806 07:49:30.629154 30920 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:49:30.508272 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:40319 (local address 127.20.202.254:42007)
0806 07:49:30.520090 (+ 11818us) server_negotiation.cc:207] Beginning negotiation
0806 07:49:30.520096 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:49:30.520115 (+ 19us) server_negotiation.cc:408] Connection header received
0806 07:49:30.520175 (+ 60us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:49:30.520181 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:49:30.520260 (+ 79us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:49:30.520418 (+ 158us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:49:30.521971 (+ 1553us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:49:30.523640 (+ 1669us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:49:30.609424 (+ 85784us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:49:30.610960 (+ 1536us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:49:30.628140 (+ 17180us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:40319: BlockingRecv error: recv got EOF from 127.20.202.193:40319 (error 108)
Metrics: {"server-negotiator.queue_time_us":11617,"thread_start_us":11426,"threads_started":1}
I20260806 07:49:30.636229 30360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:30.722127 30367 heartbeater.cc:499] Master 127.20.202.253:34181 was elected leader, sending a full tablet report...
I20260806 07:49:30.723984 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30237
W20260806 07:49:30.742853 30012 connection.cc:573] server connection from 127.20.202.194:56577 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:49:30.742865 30528 connection.cc:573] server connection from 127.20.202.194:38255 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:49:30.742872 29931 connection.cc:573] server connection from 127.20.202.194:37267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:49:30.743242 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30372
I20260806 07:49:30.751614 30496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:30.838997 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30372
I20260806 07:49:30.854909 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30509
I20260806 07:49:30.861339 30572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:30.970907 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30509
I20260806 07:49:30.995515 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29898
I20260806 07:49:30.996706 29975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:31.107326 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29898
I20260806 07:49:31.129899 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29982
I20260806 07:49:31.131161 30060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:31.238349 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29982
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestBasicOperation/3 (38728 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/4
2026-08-06T07:49:31Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:49:31Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:49:38: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:49:47.376157 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
W20260806 07:49:47.714385 31217 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:47.714700 31217 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:47.714780 31217 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:47.723366 31217 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:47.723465 31217 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:47.723532 31217 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:47.723582 31217 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:47.733726 31217 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:47.735360 31217 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:47.737175 31217 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:47.746183 31223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:47.746229 31222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:47.747197 31225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:47.747710 31217 server_base.cc:1061] running on GCE node
I20260806 07:49:47.748450 31217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:47.749922 31217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:47.751173 31217 hybrid_clock.cc:648] HybridClock initialized: now 1786002587751123 us; error 38 us; skew 500 ppm
I20260806 07:49:47.751561 31217 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:47.754066 31217 webserver.cc:533] Webserver started at http://127.20.202.254:42401/ using document root <none> and password file <none>
I20260806 07:49:47.754809 31217 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:47.754956 31217 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:47.755317 31217 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:47.757774 31217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
format_stamp: "Formatted at 2026-08-06 07:49:47 on dist-test-slave-g350"
I20260806 07:49:47.758581 31217 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
format_stamp: "Formatted at 2026-08-06 07:49:47 on dist-test-slave-g350"
I20260806 07:49:47.763868 31217 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:49:47.767405 31231 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:47.768784 31217 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:49:47.768982 31217 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
format_stamp: "Formatted at 2026-08-06 07:49:47 on dist-test-slave-g350"
I20260806 07:49:47.769171 31217 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:47.799903 31217 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:47.800837 31217 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:47.801110 31217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:47.822580 31217 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45023
I20260806 07:49:47.822620 31282 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45023 every 8 connection(s)
I20260806 07:49:47.824259 31217 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:49:47.828891 31283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:49:47.830433 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31217
I20260806 07:49:47.830669 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:49:47.832152 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37709
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
I20260806 07:49:47.838107 31283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:47.859098 31283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:49:47.860584 31233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111)
W20260806 07:49:47.862517 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37709: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111)
I20260806 07:49:47.911957 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } attempt: 1
W20260806 07:49:47.914354 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37709: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111)
I20260806 07:49:47.982669 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } attempt: 2
W20260806 07:49:47.984844 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37709: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111)
I20260806 07:49:48.076144 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } attempt: 3
W20260806 07:49:48.078444 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37709: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111)
W20260806 07:49:48.178012 31285 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:48.178318 31285 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:48.178386 31285 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:48.186533 31285 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:48.186616 31285 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:48.186668 31285 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:48.186717 31285 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:48.196920 31285 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:48.198521 31285 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:48.200306 31285 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:48.208786 31294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:48.209030 31293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:48.209828 31296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:48.210258 31285 server_base.cc:1061] running on GCE node
I20260806 07:49:48.211041 31285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:48.212468 31285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:48.213701 31285 hybrid_clock.cc:648] HybridClock initialized: now 1786002588213637 us; error 74 us; skew 500 ppm
I20260806 07:49:48.214067 31285 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:48.216348 31285 webserver.cc:533] Webserver started at http://127.20.202.253:44285/ using document root <none> and password file <none>
I20260806 07:49:48.217097 31285 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:48.217235 31285 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:48.217759 31285 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:48.221714 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } attempt: 4
I20260806 07:49:48.222271 31285 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "528f43c399224da1be4b94f6f7ca0234"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:49:48.223366 31285 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "528f43c399224da1be4b94f6f7ca0234"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
W20260806 07:49:48.223817 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37709: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111)
I20260806 07:49:48.229017 31285 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 07:49:48.232457 31303 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:48.233911 31285 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:48.234098 31285 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "528f43c399224da1be4b94f6f7ca0234"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:49:48.234308 31285 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:48.288352 31285 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:48.289287 31285 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:48.289613 31285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:48.310829 31285 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:37709
I20260806 07:49:48.310848 31354 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:37709 every 8 connection(s)
I20260806 07:49:48.312517 31285 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:49:48.316915 31355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:49:48.318454 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31285
I20260806 07:49:48.318673 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:49:48.320163 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:37635
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
I20260806 07:49:48.325825 31355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:48.340512 31355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:48.350020 31355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:49:48.351080 31305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
W20260806 07:49:48.352738 31355 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.402118 31355 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 1
W20260806 07:49:48.404570 31355 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.472941 31355 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 2
W20260806 07:49:48.475306 31355 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.523090 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } attempt: 5
I20260806 07:49:48.530119 31283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:49:48.532058 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.566579 31355 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 3
W20260806 07:49:48.568725 31355 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.583268 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 1
W20260806 07:49:48.585446 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.653702 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 2
W20260806 07:49:48.656039 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
W20260806 07:49:48.668176 31357 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:48.668503 31357 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:48.668639 31357 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:48.677263 31357 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:48.677351 31357 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:48.677451 31357 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:48.677515 31357 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:48.687763 31357 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:48.689438 31357 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:48.691303 31357 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:48.699602 31366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:48.699569 31365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:48.699914 31368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:48.700556 31357 server_base.cc:1061] running on GCE node
I20260806 07:49:48.701318 31357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:48.702806 31357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:48.704039 31357 hybrid_clock.cc:648] HybridClock initialized: now 1786002588704004 us; error 62 us; skew 500 ppm
I20260806 07:49:48.704386 31357 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:48.706705 31357 webserver.cc:533] Webserver started at http://127.20.202.252:43633/ using document root <none> and password file <none>
I20260806 07:49:48.707449 31357 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:48.707603 31357 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:48.707979 31357 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:48.710557 31357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "681764fde77049e4944a3bc5f7dba68b"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:49:48.711365 31357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "681764fde77049e4944a3bc5f7dba68b"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:49:48.711971 31355 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 4
W20260806 07:49:48.714143 31355 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.717235 31357 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:49:48.720685 31374 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:48.722098 31357 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:48.722239 31357 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "681764fde77049e4944a3bc5f7dba68b"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:49:48.722417 31357 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:48.747103 31357 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:48.747969 31357 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:48.748205 31357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:48.762290 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 3
W20260806 07:49:48.764426 31283 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111)
I20260806 07:49:48.769574 31357 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:37635
I20260806 07:49:48.769582 31426 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:37635 every 8 connection(s)
I20260806 07:49:48.771278 31357 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:49:48.775835 31427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:49:48.776237 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31357
I20260806 07:49:48.776459 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:49:48.783267 31427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:48.799525 31427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:48.806149 31427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:49:48.816979 31427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Bootstrap starting.
I20260806 07:49:48.820266 31427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Neither blocks nor log segments found. Creating new log.
I20260806 07:49:48.821389 31427 log.cc:826] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:48.824535 31427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: No bootstrap required, opened a new log
I20260806 07:49:48.832324 31427 raft_consensus.cc:359] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:48.832679 31427 raft_consensus.cc:385] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:49:48.832789 31427 raft_consensus.cc:740] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681764fde77049e4944a3bc5f7dba68b, State: Initialized, Role: FOLLOWER
I20260806 07:49:48.833468 31427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:48.834460 31433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:48.834754 31433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:48.835724 31427 sys_catalog.cc:565] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:49:48.846570 31445 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:49:48.846798 31445 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:49:48.843132 31357 master_runner.cc:187] Error getting master registration for 127.20.202.254:45023: OK, instance_id { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" instance_seqno: 1786002587816429 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:49:48.848672 31357 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:48.941687 31283 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 4
I20260806 07:49:48.952746 31283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap starting.
I20260806 07:49:48.956153 31283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Neither blocks nor log segments found. Creating new log.
I20260806 07:49:48.957268 31283 log.cc:826] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:48.959785 31283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: No bootstrap required, opened a new log
I20260806 07:49:48.964890 31283 raft_consensus.cc:359] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:48.965185 31283 raft_consensus.cc:385] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:49:48.965322 31283 raft_consensus.cc:740] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e718584a9554459ea5b8a7cdc3fe1a2f, State: Initialized, Role: FOLLOWER
I20260806 07:49:48.966022 31283 consensus_queue.cc:260] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:48.967113 31448 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:48.967437 31448 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:48.968248 31283 sys_catalog.cc:565] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:49:48.981297 31459 catalog_manager.cc:1594] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:49:48.981523 31459 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:49:48.980669 31217 master_runner.cc:187] Error getting master registration for 127.20.202.253:37709: OK, instance_id { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" instance_seqno: 1786002588304951 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:49:48.985153 31217 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:49:49.000361 31380 tablet.cc:2406] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:49:49.013371 31355 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } attempt: 5
I20260806 07:49:49.023433 31355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Bootstrap starting.
I20260806 07:49:49.026681 31355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Neither blocks nor log segments found. Creating new log.
I20260806 07:49:49.027994 31355 log.cc:826] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:49.030584 31355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: No bootstrap required, opened a new log
I20260806 07:49:49.038076 31355 raft_consensus.cc:359] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:49.038362 31355 raft_consensus.cc:385] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:49:49.038532 31355 raft_consensus.cc:740] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 528f43c399224da1be4b94f6f7ca0234, State: Initialized, Role: FOLLOWER
I20260806 07:49:49.039335 31355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:49.040419 31462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.040786 31462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:49.041682 31355 sys_catalog.cc:565] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:49:49.042081 31309 tablet.cc:2406] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:49:49.054092 31237 tablet.cc:2406] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:49:49.067006 31285 master_runner.cc:264] No leader master found from master 528f43c399224da1be4b94f6f7ca0234
I20260806 07:49:49.067235 31285 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:49:49.070418 31474 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:49:49.070677 31474 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:49:49.137104 31462 raft_consensus.cc:493] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:49:49.137441 31462 raft_consensus.cc:515] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:49.139410 31462 leader_election.cc:290] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e718584a9554459ea5b8a7cdc3fe1a2f (127.20.202.254:45023), 681764fde77049e4944a3bc5f7dba68b (127.20.202.252:37635)
I20260806 07:49:49.140791 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" is_pre_election: true
I20260806 07:49:49.141351 31258 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 528f43c399224da1be4b94f6f7ca0234 in term 0.
I20260806 07:49:49.142683 31401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681764fde77049e4944a3bc5f7dba68b" is_pre_election: true
I20260806 07:49:49.143213 31401 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 528f43c399224da1be4b94f6f7ca0234 in term 0.
I20260806 07:49:49.144136 31305 leader_election.cc:304] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 528f43c399224da1be4b94f6f7ca0234, 681764fde77049e4944a3bc5f7dba68b; no voters:
I20260806 07:49:49.149376 31462 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:49:49.149544 31462 raft_consensus.cc:493] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:49:49.149757 31462 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:49:49.153460 31462 raft_consensus.cc:515] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:49.154729 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
I20260806 07:49:49.154959 31258 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:49:49.158694 31258 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 528f43c399224da1be4b94f6f7ca0234 in term 1.
I20260806 07:49:49.159266 31307 leader_election.cc:304] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 528f43c399224da1be4b94f6f7ca0234, e718584a9554459ea5b8a7cdc3fe1a2f; no voters:
I20260806 07:49:49.164271 31401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681764fde77049e4944a3bc5f7dba68b"
I20260806 07:49:49.164506 31401 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:49:49.168342 31401 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 528f43c399224da1be4b94f6f7ca0234 in term 1.
I20260806 07:49:49.169451 31462 leader_election.cc:290] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 1 election: Requested vote from peers e718584a9554459ea5b8a7cdc3fe1a2f (127.20.202.254:45023), 681764fde77049e4944a3bc5f7dba68b (127.20.202.252:37635)
I20260806 07:49:49.169946 31462 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:49:49.170420 31462 raft_consensus.cc:697] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 LEADER]: Becoming Leader. State: Replica: 528f43c399224da1be4b94f6f7ca0234, State: Running, Role: LEADER
I20260806 07:49:49.171170 31462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:49.196525 31476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.196907 31476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:49.206611 31478 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:49:49.209244 31478 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:49:49.229326 31401 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 1 FOLLOWER]: Refusing update from remote peer 528f43c399224da1be4b94f6f7ca0234: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:49:49.230540 31476 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:49:49.237913 31258 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Refusing update from remote peer 528f43c399224da1be4b94f6f7ca0234: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:49:49.238960 31476 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:49:49.262967 31433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.263268 31433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:49.274462 31479 mvcc.cc:204] Tried to move back new op lower bound from 7315466605471625216 to 7315466605275611136. Current Snapshot: MvccSnapshot[applied={T|T < 7315466605471625216}]
I20260806 07:49:49.278955 31448 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.279249 31448 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:49.292363 31433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.292630 31433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:49.301302 31448 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.301538 31448 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:49.303176 31462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.303467 31462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:49.303836 31462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:49.304064 31462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:49.306035 31481 mvcc.cc:204] Tried to move back new op lower bound from 7315466605471625216 to 7315466605275611136. Current Snapshot: MvccSnapshot[applied={T|T < 7315466605471625216 or (T in {7315466605471625216})}]
I20260806 07:49:49.306933 31478 catalog_manager.cc:1383] Generated new cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:49:49.307049 31478 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:49:49.379977 31478 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:49:49.381807 31478 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:49:49.399788 31478 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Generated new TSK 0
I20260806 07:49:49.400990 31478 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:49:49.416887 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:49:49.819433 31487 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:49.819729 31487 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:49.819881 31487 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:49.828433 31487 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:49.828666 31487 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:49.838757 31487 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:49.840536 31487 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:49.842433 31487 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:49:49.849063 31445 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:49:49.849176 31445 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: loading cluster ID for follower catalog manager: success
W20260806 07:49:49.852758 31493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:49.853608 31492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:49.853838 31495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:49.853947 31445 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: acquiring CA information for follower catalog manager: success
I20260806 07:49:49.855273 31487 server_base.cc:1061] running on GCE node
I20260806 07:49:49.856086 31487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:49.856240 31445 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:49.857046 31487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:49.858268 31487 hybrid_clock.cc:648] HybridClock initialized: now 1786002589858197 us; error 55 us; skew 500 ppm
I20260806 07:49:49.858613 31487 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:49.860723 31487 webserver.cc:533] Webserver started at http://127.20.202.193:40773/ using document root <none> and password file <none>
I20260806 07:49:49.861469 31487 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:49.861641 31487 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:49.862038 31487 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:49.864418 31487 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:49:49.865221 31487 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:49:49.870694 31487 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 07:49:49.874428 31501 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:49.875821 31487 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:49:49.876061 31487 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:49:49.876287 31487 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:49.899070 31487 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:49.899921 31487 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:49.900146 31487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:49.901018 31487 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:49.902659 31487 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:49.902748 31487 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:49.902822 31487 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:49.902884 31487 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:49.937923 31487 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38139
I20260806 07:49:49.938011 31614 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38139 every 8 connection(s)
I20260806 07:49:49.939616 31487 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:49.946044 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31487
I20260806 07:49:49.946214 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:49:49.952098 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:49:49.975680 31616 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:49:49.975699 31620 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:49:49.976627 31620 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:49.976622 31616 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:49.978190 31620 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:49:49.978489 31616 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:49:49.980788 31320 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:49.980875 31615 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:49:49.980993 31615 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:49.981081 31248 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:49.981513 31615 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:49:49.983251 31320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:59933
I20260806 07:49:49.983439 31459 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:49:49.983497 31459 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: loading cluster ID for follower catalog manager: success
I20260806 07:49:49.985808 31391 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:49.986970 31459 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: acquiring CA information for follower catalog manager: success
I20260806 07:49:49.988692 31459 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:49:50.309478 31626 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:50.309813 31626 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:50.309945 31626 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:50.318392 31626 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:50.318616 31626 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:49:50.329567 31626 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:50.331236 31626 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:50.333061 31626 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:50.342888 31632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:50.342952 31633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:50.343572 31635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:50.344210 31626 server_base.cc:1061] running on GCE node
I20260806 07:49:50.344820 31626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:50.345849 31626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:50.347100 31626 hybrid_clock.cc:648] HybridClock initialized: now 1786002590347058 us; error 34 us; skew 500 ppm
I20260806 07:49:50.347425 31626 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:50.349761 31626 webserver.cc:533] Webserver started at http://127.20.202.194:37853/ using document root <none> and password file <none>
I20260806 07:49:50.350528 31626 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:50.350673 31626 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:50.351094 31626 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:50.353564 31626 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "10860d617e8f43f190937af389a7b215"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:49:50.354404 31626 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "10860d617e8f43f190937af389a7b215"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:49:50.359937 31626 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:49:50.363366 31641 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:50.364804 31626 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 07:49:50.364992 31626 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "10860d617e8f43f190937af389a7b215"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:49:50.365252 31626 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:50.379545 31626 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:50.380409 31626 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:50.380699 31626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:50.381842 31626 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:50.383569 31626 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:50.383661 31626 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:50.383811 31626 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:50.383911 31626 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:50.419526 31626 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:33419
I20260806 07:49:50.419557 31757 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:33419 every 8 connection(s)
I20260806 07:49:50.421000 31626 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:49:50.430056 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31626
I20260806 07:49:50.430284 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:49:50.434759 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:49:50.438294 31758 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:49:50.438686 31758 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:50.439535 31759 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:49:50.439688 31759 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:50.439685 31758 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:49:50.440268 31759 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:49:50.441813 31248 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:33419)
I20260806 07:49:50.442610 31391 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:33419)
I20260806 07:49:50.443998 31760 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:49:50.444144 31760 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:50.444541 31760 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:49:50.445549 31320 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:33419)
I20260806 07:49:50.446465 31320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:35889
W20260806 07:49:50.775261 31765 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:50.775622 31765 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:50.775771 31765 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:50.784960 31765 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:50.785185 31765 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:49:50.795265 31765 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:50.796924 31765 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:50.798900 31765 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:50.808250 31770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:50.808279 31771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:50.809461 31773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:50.810544 31765 server_base.cc:1061] running on GCE node
I20260806 07:49:50.811113 31765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:50.812012 31765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:50.813220 31765 hybrid_clock.cc:648] HybridClock initialized: now 1786002590813180 us; error 28 us; skew 500 ppm
I20260806 07:49:50.813565 31765 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:50.815766 31765 webserver.cc:533] Webserver started at http://127.20.202.195:45391/ using document root <none> and password file <none>
I20260806 07:49:50.816540 31765 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:50.816699 31765 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:50.817077 31765 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:49:50.819716 31765 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "ddc6ef5af7a843d5b9aadb3613c579ac"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:49:50.820488 31765 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "ddc6ef5af7a843d5b9aadb3613c579ac"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:49:50.826251 31765 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:49:50.830092 31779 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:50.831635 31765 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:49:50.831832 31765 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "ddc6ef5af7a843d5b9aadb3613c579ac"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:49:50.832072 31765 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:50.856571 31765 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:50.857532 31765 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:50.857851 31765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:50.858871 31765 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:50.860594 31765 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:50.860728 31765 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:50.860854 31765 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:50.860950 31765 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:50.898391 31765 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:37041
I20260806 07:49:50.898478 31893 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:37041 every 8 connection(s)
I20260806 07:49:50.899852 31765 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:49:50.901038 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31765
I20260806 07:49:50.901226 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:49:50.918179 31894 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:49:50.918306 31896 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:49:50.918612 31896 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:50.918607 31894 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:50.919768 31896 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:49:50.919832 31894 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:49:50.920871 31895 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:49:50.921030 31895 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:50.921576 31895 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:49:50.921764 31391 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:37041)
I20260806 07:49:50.921779 31320 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:37041)
I20260806 07:49:50.922729 31248 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:37041)
I20260806 07:49:50.922838 31320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:49651
I20260806 07:49:50.928460 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:49:50.935985 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:49:50.936182 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31217
I20260806 07:49:50.937268 31278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:50.986918 31620 heartbeater.cc:499] Master 127.20.202.253:37709 was elected leader, sending a full tablet report...
I20260806 07:49:51.051767 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31217
I20260806 07:49:51.075559 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--webserver_interface=127.20.202.254
--webserver_port=42401
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
W20260806 07:49:51.146772 31307 consensus_peers.cc:597] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 -> Peer e718584a9554459ea5b8a7cdc3fe1a2f (127.20.202.254:45023): Couldn't send request to peer e718584a9554459ea5b8a7cdc3fe1a2f. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:45023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:49:51.418711 31903 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:51.419094 31903 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:51.419246 31903 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:51.428069 31903 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:49:51.428193 31903 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:51.428314 31903 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:49:51.428396 31903 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:49:51.438944 31903 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=42401
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:51.440738 31903 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:51.442677 31903 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:51.445752 31759 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:45023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:45023: connect: Connection refused (error 111)
I20260806 07:49:51.448643 31760 heartbeater.cc:499] Master 127.20.202.253:37709 was elected leader, sending a full tablet report...
W20260806 07:49:51.451416 31911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:51.453125 31914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:51.453282 31912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:51.453938 31903 server_base.cc:1061] running on GCE node
I20260806 07:49:51.454692 31903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:51.456116 31903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:51.457351 31903 hybrid_clock.cc:648] HybridClock initialized: now 1786002591457305 us; error 49 us; skew 500 ppm
I20260806 07:49:51.457765 31903 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:51.460083 31903 webserver.cc:533] Webserver started at http://127.20.202.254:42401/ using document root <none> and password file <none>
I20260806 07:49:51.460839 31903 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:51.461004 31903 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:51.466595 31903 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:49:51.469725 31920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:51.471266 31903 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:51.471450 31903 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
format_stamp: "Formatted at 2026-08-06 07:49:47 on dist-test-slave-g350"
I20260806 07:49:51.472173 31903 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:51.496794 31903 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:51.497771 31903 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:51.498093 31903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:51.519577 31903 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45023
I20260806 07:49:51.519595 31972 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45023 every 8 connection(s)
I20260806 07:49:51.521284 31903 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:49:51.527837 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31903
I20260806 07:49:51.528160 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31487
I20260806 07:49:51.529034 31973 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:49:51.533689 31973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap starting.
I20260806 07:49:51.538175 31610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:51.553290 31973 log.cc:826] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:49:51.562081 31973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:49:51.562713 31973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap complete.
I20260806 07:49:51.569501 31973 raft_consensus.cc:359] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:51.570309 31973 raft_consensus.cc:740] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e718584a9554459ea5b8a7cdc3fe1a2f, State: Initialized, Role: FOLLOWER
I20260806 07:49:51.571173 31973 consensus_queue.cc:260] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:49:51.572067 31979 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:51.572377 31979 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:51.573042 31973 sys_catalog.cc:565] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:49:51.584194 31990 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:49:51.584313 31990 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: loading cluster ID for follower catalog manager: success
I20260806 07:49:51.589620 31990 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: acquiring CA information for follower catalog manager: success
I20260806 07:49:51.592847 31990 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:49:51.647034 31476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:51.647382 31476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:51.659421 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31487
I20260806 07:49:51.667333 31979 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:51.667652 31979 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:49:51.680584 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38139
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--webserver_port=40773
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:49:51.925266 31896 heartbeater.cc:499] Master 127.20.202.253:37709 was elected leader, sending a full tablet report...
I20260806 07:49:51.936004 31937 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ddc6ef5af7a843d5b9aadb3613c579ac" instance_seqno: 1786002590890063) as {username='slave'} at 127.20.202.195:53737; Asking this server to re-register.
I20260806 07:49:51.937028 31895 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:51.939275 31937 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:37041)
W20260806 07:49:52.060424 32000 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:52.060788 32000 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:52.060945 32000 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:52.069372 32000 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:52.069620 32000 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:52.080475 32000 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38139
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40773
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:52.082171 32000 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:52.083994 32000 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:52.093930 32006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:52.093962 32007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:52.094328 32009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:52.095530 32000 server_base.cc:1061] running on GCE node
I20260806 07:49:52.096077 32000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:52.097074 32000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:52.098268 32000 hybrid_clock.cc:648] HybridClock initialized: now 1786002592098217 us; error 48 us; skew 500 ppm
I20260806 07:49:52.098541 32000 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:52.100694 32000 webserver.cc:533] Webserver started at http://127.20.202.193:40773/ using document root <none> and password file <none>
I20260806 07:49:52.101511 32000 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:52.101689 32000 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:52.107064 32000 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 07:49:52.109982 32015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:52.111470 32000 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:52.111660 32000 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:49:52.112389 32000 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:52.124457 32000 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:52.125286 32000 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:52.125586 32000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:52.126590 32000 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:52.128357 32000 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:52.128468 32000 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:52.128614 32000 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:52.128688 32000 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:52.165261 32000 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38139
I20260806 07:49:52.165350 32129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38139 every 8 connection(s)
I20260806 07:49:52.166890 32000 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:52.176987 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32000
I20260806 07:49:52.184849 32132 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:49:52.184883 32131 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:49:52.185372 32132 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:52.185370 32131 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:52.186807 32132 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:49:52.186842 31476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:49:52.187135 31476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:49:52.188524 32131 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:49:52.189126 31320 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:52.189723 31937 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:52.190479 31320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:49155
I20260806 07:49:52.197647 32130 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:49:52.197826 32130 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:52.198257 32130 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:49:52.199491 31391 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:52.221999 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32000
I20260806 07:49:52.237375 32125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:52.331935 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32000
I20260806 07:49:52.350715 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38139
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--webserver_port=40773
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260806 07:49:52.453964 31759 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:49:52.455125 31937 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "10860d617e8f43f190937af389a7b215" instance_seqno: 1786002590411274) as {username='slave'} at 127.20.202.194:38145; Asking this server to re-register.
I20260806 07:49:52.455687 31759 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:52.456833 31937 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:33419)
W20260806 07:49:52.709844 32146 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:52.710131 32146 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:52.710213 32146 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:49:52.710273 32146 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:52.718593 32146 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:52.718770 32146 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:52.728600 32146 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38139
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40773
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:52.730325 32146 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:52.732098 32146 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:52.742224 32152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:52.742223 32153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:52.742904 32155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:52.743618 32146 server_base.cc:1061] running on GCE node
I20260806 07:49:52.744238 32146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:52.745074 32146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:52.746287 32146 hybrid_clock.cc:648] HybridClock initialized: now 1786002592746213 us; error 56 us; skew 500 ppm
I20260806 07:49:52.746583 32146 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:52.748647 32146 webserver.cc:533] Webserver started at http://127.20.202.193:40773/ using document root <none> and password file <none>
I20260806 07:49:52.749378 32146 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:52.749523 32146 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:52.755148 32146 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:49:52.758229 32161 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:52.759558 32146 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:49:52.759727 32146 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:49:52.760535 32146 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:52.778944 32146 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:52.779819 32146 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:52.780099 32146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:52.781104 32146 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:52.782780 32146 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:52.782903 32146 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:52.783025 32146 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:52.783118 32146 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:49:52.783489 32146 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:49:52.783603 32146 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:49:52.783686 32146 fault_injection.cc:59] injecting fault for kudu (pid 32146): FLAGS_fault_before_start (process will exit)
I20260806 07:49:52.805250 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38139
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--webserver_port=40773
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:49:53.157964 32169 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:49:53.158339 32169 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:49:53.158463 32169 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:49:53.167517 32169 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:49:53.167765 32169 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:49:53.178871 32169 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38139
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40773
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:49:53.180619 32169 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:49:53.182592 32169 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:49:53.192946 32176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:53.192914 32175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:49:53.193737 32178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:49:53.194355 32169 server_base.cc:1061] running on GCE node
I20260806 07:49:53.195006 32169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:49:53.196050 32169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:49:53.197312 32169 hybrid_clock.cc:648] HybridClock initialized: now 1786002593197258 us; error 61 us; skew 500 ppm
I20260806 07:49:53.197741 32169 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:49:53.200016 32169 webserver.cc:533] Webserver started at http://127.20.202.193:40773/ using document root <none> and password file <none>
I20260806 07:49:53.200862 32169 fs_manager.cc:362] Metadata directory not provided
I20260806 07:49:53.201031 32169 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:49:53.206983 32169 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:49:53.209923 32184 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:53.211345 32169 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:49:53.211555 32169 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:49:53.212332 32169 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:49:53.232744 32169 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:49:53.233783 32169 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:49:53.234115 32169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:49:53.235177 32169 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:49:53.237957 32169 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:49:53.238065 32169 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:53.238219 32169 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:49:53.238296 32169 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:49:53.276448 32169 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38139
I20260806 07:49:53.276467 32298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38139 every 8 connection(s)
I20260806 07:49:53.278344 32169 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:49:53.278869 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32169
I20260806 07:49:53.279261 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32169
I20260806 07:49:53.302846 32300 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:49:53.303369 32300 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:53.304630 32300 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:49:53.307376 31937 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:53.307624 32299 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:49:53.307749 32299 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:53.308230 32299 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:49:53.309378 31391 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:53.310252 32294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:53.399154 32301 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:49:53.399464 32301 heartbeater.cc:461] Registering TS with master...
I20260806 07:49:53.400283 32301 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:49:53.400771 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32169
I20260806 07:49:53.401925 31320 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:38139)
I20260806 07:49:53.403209 31320 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:44067
W20260806 07:49:53.421241 31305 connection.cc:573] server connection from 127.20.202.193:44067 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:49:53.421402 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31626
I20260806 07:49:53.430053 31753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:53.518589 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31626
W20260806 07:49:53.541497 31306 connection.cc:573] server connection from 127.20.202.194:35889 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:49:53.541564 31376 connection.cc:573] server connection from 127.20.202.194:41937 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:49:53.542215 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31765
I20260806 07:49:53.551151 31889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:53.638576 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31765
I20260806 07:49:53.657003 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31903
I20260806 07:49:53.663756 31968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:53.769395 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31903
I20260806 07:49:53.797756 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31285
I20260806 07:49:53.799285 31350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:53.850574 31357 master_runner.cc:187] Error getting master registration for 127.20.202.254:45023: Network error: Client connection negotiation failed: client connection to 127.20.202.254:45023: connect: Connection refused (error 111),
I20260806 07:49:53.850803 31357 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:49:53.910360 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31285
I20260806 07:49:53.933059 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31357
I20260806 07:49:53.934449 31422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:49:54.045993 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31357
Shutting down hive metastore.
2026-08-06T07:49:54Z chronyd exiting
2026-08-06T07:49:54Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:49:54Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:49:55: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:50:03.719274 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
W20260806 07:50:04.056354 32508 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:04.056735 32508 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:04.056881 32508 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:04.065380 32508 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:04.065510 32508 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:04.065662 32508 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:04.065759 32508 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:04.076031 32508 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:04.077749 32508 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:04.079644 32508 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:04.088107 32513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:04.088353 32514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:04.088553 32516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:04.089561 32508 server_base.cc:1061] running on GCE node
I20260806 07:50:04.090437 32508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:04.091799 32508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:04.093044 32508 hybrid_clock.cc:648] HybridClock initialized: now 1786002604092985 us; error 56 us; skew 500 ppm
I20260806 07:50:04.093393 32508 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:04.095809 32508 webserver.cc:533] Webserver started at http://127.20.202.254:44205/ using document root <none> and password file <none>
I20260806 07:50:04.096541 32508 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:04.096688 32508 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:04.102182 32508 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 07:50:04.105218 32522 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:04.106652 32508 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:04.106837 32508 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
format_stamp: "Formatted at 2026-08-06 07:49:47 on dist-test-slave-g350"
I20260806 07:50:04.107569 32508 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:04.126523 32508 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:04.127573 32508 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:04.127872 32508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:04.149816 32508 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45023
I20260806 07:50:04.149852 32575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45023 every 8 connection(s)
I20260806 07:50:04.151551 32508 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:04.158756 32576 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:04.161926 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32508
I20260806 07:50:04.162112 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:50:04.162213 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap starting.
I20260806 07:50:04.163134 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37709
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
I20260806 07:50:04.178886 32576 log.cc:826] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:04.188091 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:04.188632 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap complete.
I20260806 07:50:04.195385 32576 raft_consensus.cc:359] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:04.196121 32576 raft_consensus.cc:740] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e718584a9554459ea5b8a7cdc3fe1a2f, State: Initialized, Role: FOLLOWER
I20260806 07:50:04.196802 32576 consensus_queue.cc:260] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:04.197754 32580 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:04.198068 32580 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:04.198832 32576 sys_catalog.cc:565] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:04.209300 32591 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:50:04.209538 32591 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: loading cluster ID for follower catalog manager: success
I20260806 07:50:04.214669 32591 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: acquiring CA information for follower catalog manager: success
I20260806 07:50:04.217324 32591 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:04.230242 32508 master_runner.cc:187] Error getting master registration for 127.20.202.253:37709: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37709: connect: Connection refused (error 111),
I20260806 07:50:04.234282 32508 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:04.533785 32578 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:04.534106 32578 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:04.534272 32578 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:04.542799 32578 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:04.542909 32578 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:04.542989 32578 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:04.543061 32578 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:04.553392 32578 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:04.555220 32578 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:04.557145 32578 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:04.565320 32600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:04.565446 32599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:04.567078 32602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:04.567798 32578 server_base.cc:1061] running on GCE node
I20260806 07:50:04.568593 32578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:04.570123 32578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:04.571372 32578 hybrid_clock.cc:648] HybridClock initialized: now 1786002604571317 us; error 41 us; skew 500 ppm
I20260806 07:50:04.571781 32578 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:04.574225 32578 webserver.cc:533] Webserver started at http://127.20.202.253:41179/ using document root <none> and password file <none>
I20260806 07:50:04.575013 32578 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:04.575176 32578 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:04.580991 32578 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:50:04.584121 32608 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:04.585779 32578 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:04.586021 32578 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "528f43c399224da1be4b94f6f7ca0234"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:50:04.586746 32578 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:04.611213 32578 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:04.612164 32578 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:04.612534 32578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:04.634371 32578 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:37709
I20260806 07:50:04.634397 32660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:37709 every 8 connection(s)
I20260806 07:50:04.636097 32578 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:50:04.636503 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32578
I20260806 07:50:04.636722 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:50:04.638010 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:37635
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
I20260806 07:50:04.643649 32661 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:04.646997 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Bootstrap starting.
I20260806 07:50:04.663558 32661 log.cc:826] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:04.672123 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:04.672680 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Bootstrap complete.
I20260806 07:50:04.679399 32661 raft_consensus.cc:359] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:04.680095 32661 raft_consensus.cc:740] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 528f43c399224da1be4b94f6f7ca0234, State: Initialized, Role: FOLLOWER
I20260806 07:50:04.680893 32661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:04.681821 32665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:04.682229 32665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:04.683065 32661 sys_catalog.cc:565] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:04.698933 32676 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:50:04.699121 32676 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: loading cluster ID for follower catalog manager: success
I20260806 07:50:04.703137 32676 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: acquiring CA information for follower catalog manager: success
I20260806 07:50:04.705816 32676 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:04.730777 32578 master_runner.cc:187] Error getting master registration for 127.20.202.252:37635: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37635: connect: Connection refused (error 111),
I20260806 07:50:04.736304 32578 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:05.005155 32663 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:05.005524 32663 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:05.005702 32663 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:05.014319 32663 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:05.014412 32663 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:05.014529 32663 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:05.014642 32663 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:05.024971 32663 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:05.026690 32663 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:05.028594 32663 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:05.036902 32684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:05.037917 32685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:05.038120 32687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:05.039420 32663 server_base.cc:1061] running on GCE node
I20260806 07:50:05.040280 32663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:05.041980 32663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:05.043228 32663 hybrid_clock.cc:648] HybridClock initialized: now 1786002605043188 us; error 37 us; skew 500 ppm
I20260806 07:50:05.043623 32663 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:05.046183 32663 webserver.cc:533] Webserver started at http://127.20.202.252:44349/ using document root <none> and password file <none>
I20260806 07:50:05.046983 32663 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:05.047137 32663 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:05.053169 32663 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:50:05.056499 32693 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:05.058179 32663 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:50:05.058363 32663 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "681764fde77049e4944a3bc5f7dba68b"
format_stamp: "Formatted at 2026-08-06 07:49:48 on dist-test-slave-g350"
I20260806 07:50:05.059108 32663 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:05.094199 32663 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:05.095146 32663 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:05.095470 32663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:05.118633 32663 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:37635
I20260806 07:50:05.118657 32745 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:37635 every 8 connection(s)
I20260806 07:50:05.120479 32663 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:50:05.122073 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32663
I20260806 07:50:05.122272 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:50:05.129897 32746 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:05.133844 32746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Bootstrap starting.
I20260806 07:50:05.152920 32746 log.cc:826] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:05.162402 32746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:05.163144 32746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: Bootstrap complete.
I20260806 07:50:05.170207 32746 raft_consensus.cc:359] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:05.170957 32746 raft_consensus.cc:740] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681764fde77049e4944a3bc5f7dba68b, State: Initialized, Role: FOLLOWER
I20260806 07:50:05.171590 32746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:05.172380 32751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:05.172657 32751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:05.173455 32746 sys_catalog.cc:565] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:05.184810 32762 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:50:05.184901 32762 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: loading cluster ID for follower catalog manager: success
I20260806 07:50:05.189920 32762 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: acquiring CA information for follower catalog manager: success
I20260806 07:50:05.192538 32762 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:05.211460 32663 master_runner.cc:264] No leader master found from master 681764fde77049e4944a3bc5f7dba68b
I20260806 07:50:05.211635 32663 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:05.501405 32765 raft_consensus.cc:493] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:50:05.501775 32765 raft_consensus.cc:515] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:05.503975 32765 leader_election.cc:290] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 528f43c399224da1be4b94f6f7ca0234 (127.20.202.253:37709), 681764fde77049e4944a3bc5f7dba68b (127.20.202.252:37635)
I20260806 07:50:05.512552 32636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "528f43c399224da1be4b94f6f7ca0234" is_pre_election: true
I20260806 07:50:05.513156 32636 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e718584a9554459ea5b8a7cdc3fe1a2f in term 1.
I20260806 07:50:05.513923 32524 leader_election.cc:304] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 528f43c399224da1be4b94f6f7ca0234, e718584a9554459ea5b8a7cdc3fe1a2f; no voters:
I20260806 07:50:05.514418 32765 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:50:05.514587 32765 raft_consensus.cc:493] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:50:05.514703 32765 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:50:05.514482 32721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "681764fde77049e4944a3bc5f7dba68b" is_pre_election: true
I20260806 07:50:05.515208 32721 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e718584a9554459ea5b8a7cdc3fe1a2f in term 1.
I20260806 07:50:05.520198 32765 raft_consensus.cc:515] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:05.521039 32765 leader_election.cc:290] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [CANDIDATE]: Term 2 election: Requested vote from peers 528f43c399224da1be4b94f6f7ca0234 (127.20.202.253:37709), 681764fde77049e4944a3bc5f7dba68b (127.20.202.252:37635)
I20260806 07:50:05.521548 32636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "528f43c399224da1be4b94f6f7ca0234"
I20260806 07:50:05.521821 32721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "681764fde77049e4944a3bc5f7dba68b"
I20260806 07:50:05.521844 32636 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:50:05.522069 32721 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:50:05.527671 32636 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e718584a9554459ea5b8a7cdc3fe1a2f in term 2.
I20260806 07:50:05.528332 32721 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e718584a9554459ea5b8a7cdc3fe1a2f in term 2.
I20260806 07:50:05.528393 32524 leader_election.cc:304] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 528f43c399224da1be4b94f6f7ca0234, e718584a9554459ea5b8a7cdc3fe1a2f; no voters:
I20260806 07:50:05.528786 32765 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:50:05.529830 32765 raft_consensus.cc:697] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 LEADER]: Becoming Leader. State: Replica: e718584a9554459ea5b8a7cdc3fe1a2f, State: Running, Role: LEADER
I20260806 07:50:05.531081 32765 consensus_queue.cc:237] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:05.534840 301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e718584a9554459ea5b8a7cdc3fe1a2f. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:05.535123 301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:05.536734 303 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:05.538928 303 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:05.539819 303 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:50:05.539899 303 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:05.541146 303 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:05.542071 303 catalog_manager.cc:6081] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Loaded TSK: 0
I20260806 07:50:05.543025 303 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:50:05.568171 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:50:05.910414 304 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:05.910727 304 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:05.910858 304 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:05.919310 304 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:05.919509 304 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:05.929461 304 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:05.931286 304 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:05.933142 304 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:05.943524 309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:05.943475 310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:05.946382 312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:05.946513 304 server_base.cc:1061] running on GCE node
I20260806 07:50:05.947593 304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:05.948910 304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:05.950168 304 hybrid_clock.cc:648] HybridClock initialized: now 1786002605950102 us; error 59 us; skew 500 ppm
I20260806 07:50:05.950541 304 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:05.952699 304 webserver.cc:533] Webserver started at http://127.20.202.193:41185/ using document root <none> and password file <none>
I20260806 07:50:05.953516 304 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:05.953708 304 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:05.959545 304 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:50:05.962471 318 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:05.963953 304 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:05.964134 304 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:50:05.964957 304 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:05.981913 304 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:05.982764 304 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:05.983080 304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:05.984200 304 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:05.985968 304 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:05.986069 304 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:05.986218 304 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:05.986290 304 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:06.004369 32636 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Refusing update from remote peer e718584a9554459ea5b8a7cdc3fe1a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:50:06.005528 301 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:50:06.009713 378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e718584a9554459ea5b8a7cdc3fe1a2f. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:06.010047 378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:06.010017 32721 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 2 FOLLOWER]: Refusing update from remote peer e718584a9554459ea5b8a7cdc3fe1a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:50:06.011049 32765 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:50:06.013247 301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:06.013504 301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:06.014565 378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:06.014763 378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:06.017828 386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e718584a9554459ea5b8a7cdc3fe1a2f. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:06.018059 386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:06.018186 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:06.018409 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:06.019963 386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:06.020207 386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:06.034026 304 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41191
I20260806 07:50:06.034114 439 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41191 every 8 connection(s)
I20260806 07:50:06.035683 304 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:06.043375 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 304
I20260806 07:50:06.043648 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:50:06.049950 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:06.079195 441 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:50:06.079689 441 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:06.080066 440 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:50:06.080243 440 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:06.080624 442 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:50:06.080816 442 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:06.081081 440 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:50:06.081426 442 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:50:06.081790 441 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:50:06.083730 32710 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:06.083727 32539 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:06.083936 32625 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:06.086362 32539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:48467
W20260806 07:50:06.409226 449 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:06.409545 449 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:06.409694 449 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:06.418124 449 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:06.418277 449 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:50:06.428098 449 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:06.429725 449 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:06.431543 449 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:06.441694 457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:06.441705 456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:06.442358 459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:06.442927 449 server_base.cc:1061] running on GCE node
I20260806 07:50:06.443532 449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:06.444517 449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:06.445713 449 hybrid_clock.cc:648] HybridClock initialized: now 1786002606445669 us; error 56 us; skew 500 ppm
I20260806 07:50:06.445986 449 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:06.448122 449 webserver.cc:533] Webserver started at http://127.20.202.194:41569/ using document root <none> and password file <none>
I20260806 07:50:06.448840 449 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:06.448956 449 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:06.454437 449 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:50:06.457557 465 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:06.459178 449 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:50:06.459352 449 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "10860d617e8f43f190937af389a7b215"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:50:06.460069 449 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:06.475512 449 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:06.476330 449 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:06.476619 449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:06.477634 449 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:06.479259 449 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:06.479331 449 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:06.479506 449 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:06.479557 449 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:06.514537 578 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:38871 every 8 connection(s)
I20260806 07:50:06.514567 449 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:38871
I20260806 07:50:06.516263 449 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:50:06.526360 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 449
I20260806 07:50:06.526549 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:50:06.530943 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:06.535431 580 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:50:06.535857 579 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:50:06.535864 580 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:06.536020 579 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:06.536509 581 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:50:06.536661 581 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:06.537315 579 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:50:06.537348 580 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:50:06.537376 581 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:50:06.539239 32539 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:38871)
I20260806 07:50:06.539305 32625 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:38871)
I20260806 07:50:06.539371 32710 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:38871)
I20260806 07:50:06.540397 32539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:41609
W20260806 07:50:06.885432 587 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:06.885776 587 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:06.885955 587 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:06.894539 587 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:06.894747 587 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:50:06.904866 587 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:06.906620 587 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:06.908483 587 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:06.918452 593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:06.918388 592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:06.918951 595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:06.919612 587 server_base.cc:1061] running on GCE node
I20260806 07:50:06.920149 587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:06.921026 587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:06.922228 587 hybrid_clock.cc:648] HybridClock initialized: now 1786002606922175 us; error 40 us; skew 500 ppm
I20260806 07:50:06.922536 587 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:06.924620 587 webserver.cc:533] Webserver started at http://127.20.202.195:37199/ using document root <none> and password file <none>
I20260806 07:50:06.925366 587 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:06.925519 587 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:06.931111 587 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:50:06.933831 601 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:06.935124 587 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:06.935294 587 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "ddc6ef5af7a843d5b9aadb3613c579ac"
format_stamp: "Formatted at 2026-08-06 07:49:50 on dist-test-slave-g350"
I20260806 07:50:06.935997 587 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:06.958408 587 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:06.959362 587 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:06.959676 587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:06.960686 587 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:06.962522 587 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:06.962643 587 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:06.962802 587 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:06.962881 587 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:07.000795 587 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:46585
I20260806 07:50:07.000892 714 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:46585 every 8 connection(s)
I20260806 07:50:07.002514 587 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:50:07.008126 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 587
I20260806 07:50:07.008358 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:50:07.020963 716 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:50:07.021517 716 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:07.021711 715 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:50:07.021886 715 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:07.023026 716 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:50:07.023052 715 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:50:07.024088 717 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:50:07.024272 717 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:07.024825 717 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:50:07.024837 32539 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:46585)
I20260806 07:50:07.025099 32710 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:46585)
I20260806 07:50:07.026041 32625 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:46585)
I20260806 07:50:07.026121 32539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:37533
I20260806 07:50:07.038950 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:07.048709 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:07.048933 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32508
I20260806 07:50:07.049934 32571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:07.089324 441 heartbeater.cc:499] Master 127.20.202.254:45023 was elected leader, sending a full tablet report...
I20260806 07:50:07.163950 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32508
I20260806 07:50:07.188014 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--webserver_interface=127.20.202.254
--webserver_port=44205
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635 with env {}
W20260806 07:50:07.532711 725 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:07.533028 725 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:07.533210 725 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:07.542505 725 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:07.542634 725 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:07.542796 725 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260806 07:50:07.542776 580 heartbeater.cc:499] Master 127.20.202.254:45023 was elected leader, sending a full tablet report...
W20260806 07:50:07.542858 725 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:50:07.544325 580 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:45023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:45023: connect: Connection refused (error 111)
I20260806 07:50:07.553228 725 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=44205
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:07.554971 725 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:07.556967 725 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:07.566200 732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:07.566228 731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:07.566962 734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:07.567931 725 server_base.cc:1061] running on GCE node
I20260806 07:50:07.568727 725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:07.570390 725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:07.571635 725 hybrid_clock.cc:648] HybridClock initialized: now 1786002607571572 us; error 52 us; skew 500 ppm
I20260806 07:50:07.572014 725 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:07.574687 725 webserver.cc:533] Webserver started at http://127.20.202.254:44205/ using document root <none> and password file <none>
I20260806 07:50:07.575492 725 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:07.575645 725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:07.581337 725 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 07:50:07.584472 740 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:07.585918 725 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:07.586129 725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
format_stamp: "Formatted at 2026-08-06 07:49:47 on dist-test-slave-g350"
I20260806 07:50:07.586855 725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:07.630450 725 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:07.631384 725 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:07.631705 725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:07.658619 725 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45023
I20260806 07:50:07.658623 792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45023 every 8 connection(s)
I20260806 07:50:07.660347 725 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:07.661820 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 725
I20260806 07:50:07.662158 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 304
I20260806 07:50:07.669108 793 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:07.673774 793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap starting.
I20260806 07:50:07.674019 435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:07.699123 793 log.cc:826] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:07.710207 793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:07.710845 793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: Bootstrap complete.
I20260806 07:50:07.720824 793 raft_consensus.cc:359] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:07.721807 793 raft_consensus.cc:740] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e718584a9554459ea5b8a7cdc3fe1a2f, State: Initialized, Role: FOLLOWER
I20260806 07:50:07.722646 793 consensus_queue.cc:260] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:07.723569 800 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:07.723846 800 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:07.724807 793 sys_catalog.cc:565] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:07.735510 811 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:50:07.735679 811 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: loading cluster ID for follower catalog manager: success
I20260806 07:50:07.739988 811 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: acquiring CA information for follower catalog manager: success
I20260806 07:50:07.744874 811 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:07.780218 725 master_runner.cc:264] No leader master found from master e718584a9554459ea5b8a7cdc3fe1a2f
I20260806 07:50:07.780403 725 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:07.792991 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 304
I20260806 07:50:07.812004 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41191
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--webserver_port=41185
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:08.028208 716 heartbeater.cc:499] Master 127.20.202.254:45023 was elected leader, sending a full tablet report...
I20260806 07:50:08.040930 758 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ddc6ef5af7a843d5b9aadb3613c579ac" instance_seqno: 1786002606991533) as {username='slave'} at 127.20.202.195:38135; Asking this server to re-register.
I20260806 07:50:08.042157 716 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:08.044159 758 ts_manager.cc:194] Registered new tserver with Master: ddc6ef5af7a843d5b9aadb3613c579ac (127.20.202.195:46585)
W20260806 07:50:08.171056 818 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:08.171402 818 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:08.171541 818 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:08.180558 818 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:08.180781 818 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:08.191011 818 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41191
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41185
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:08.192854 818 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:08.194765 818 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:08.204672 824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:08.204687 825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:08.205446 827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:08.206143 818 server_base.cc:1061] running on GCE node
I20260806 07:50:08.206707 818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:08.207638 818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:08.208873 818 hybrid_clock.cc:648] HybridClock initialized: now 1786002608208829 us; error 56 us; skew 500 ppm
I20260806 07:50:08.209234 818 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:08.211340 818 webserver.cc:533] Webserver started at http://127.20.202.193:41185/ using document root <none> and password file <none>
I20260806 07:50:08.212088 818 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:08.212235 818 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:08.217520 818 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:08.220132 833 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:08.221524 818 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:50:08.221736 818 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:50:08.222559 818 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:08.262884 818 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:08.263859 818 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:08.264148 818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:08.265344 818 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:08.267544 818 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:08.267629 818 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:08.267721 818 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:08.267771 818 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:08.306644 818 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41191
I20260806 07:50:08.306705 947 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41191 every 8 connection(s)
I20260806 07:50:08.308423 818 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:08.318131 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 818
I20260806 07:50:08.332857 949 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:50:08.332984 950 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:50:08.333072 948 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:50:08.333354 949 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:08.333344 948 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:08.333346 950 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:08.334661 948 heartbeater.cc:507] Master 127.20.202.252:37635 requested a full tablet report, sending...
I20260806 07:50:08.336087 949 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:50:08.336632 950 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:50:08.336941 32710 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:08.337313 758 ts_manager.cc:194] Registered new tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:08.338099 32625 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:08.432397 958 raft_consensus.cc:493] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e718584a9554459ea5b8a7cdc3fe1a2f)
I20260806 07:50:08.432781 958 raft_consensus.cc:515] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:08.451208 958 leader_election.cc:290] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e718584a9554459ea5b8a7cdc3fe1a2f (127.20.202.254:45023), 681764fde77049e4944a3bc5f7dba68b (127.20.202.252:37635)
I20260806 07:50:08.463593 32721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "681764fde77049e4944a3bc5f7dba68b" is_pre_election: true
I20260806 07:50:08.468573 768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" is_pre_election: true
I20260806 07:50:08.469156 768 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 528f43c399224da1be4b94f6f7ca0234 in term 2.
I20260806 07:50:08.469877 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 818
I20260806 07:50:08.469839 32612 leader_election.cc:304] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 528f43c399224da1be4b94f6f7ca0234, e718584a9554459ea5b8a7cdc3fe1a2f; no voters: 681764fde77049e4944a3bc5f7dba68b
I20260806 07:50:08.470727 958 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:50:08.470845 958 raft_consensus.cc:493] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e718584a9554459ea5b8a7cdc3fe1a2f)
I20260806 07:50:08.470940 958 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:50:08.476925 958 raft_consensus.cc:515] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:08.478538 768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f"
I20260806 07:50:08.478570 32721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "528f43c399224da1be4b94f6f7ca0234" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "681764fde77049e4944a3bc5f7dba68b"
I20260806 07:50:08.478767 768 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:50:08.478906 958 leader_election.cc:290] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 3 election: Requested vote from peers e718584a9554459ea5b8a7cdc3fe1a2f (127.20.202.254:45023), 681764fde77049e4944a3bc5f7dba68b (127.20.202.252:37635)
I20260806 07:50:08.482897 768 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 528f43c399224da1be4b94f6f7ca0234 in term 3.
I20260806 07:50:08.483435 32612 leader_election.cc:304] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 528f43c399224da1be4b94f6f7ca0234, e718584a9554459ea5b8a7cdc3fe1a2f; no voters: 681764fde77049e4944a3bc5f7dba68b
I20260806 07:50:08.483837 958 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:50:08.484503 958 raft_consensus.cc:697] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [term 3 LEADER]: Becoming Leader. State: Replica: 528f43c399224da1be4b94f6f7ca0234, State: Running, Role: LEADER
I20260806 07:50:08.485006 958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:08.486114 943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:08.487198 964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:08.487497 964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:08.488499 967 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:08.491351 967 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:08.492338 967 catalog_manager.cc:1295] Loaded cluster ID: d65e3332c24748c3b2c91063e34fb6f1
I20260806 07:50:08.492437 967 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:08.493580 967 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:08.494428 967 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234: Loaded TSK: 0
I20260806 07:50:08.495289 967 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:50:08.552429 580 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:50:08.552667 580 heartbeater.cc:499] Master 127.20.202.254:45023 was elected leader, sending a full tablet report...
I20260806 07:50:08.553697 758 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "10860d617e8f43f190937af389a7b215" instance_seqno: 1786002606506090) as {username='slave'} at 127.20.202.194:33209; Asking this server to re-register.
I20260806 07:50:08.554329 580 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:08.555415 758 ts_manager.cc:194] Registered new tserver with Master: 10860d617e8f43f190937af389a7b215 (127.20.202.194:38871)
I20260806 07:50:08.577751 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 818
I20260806 07:50:08.596855 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41191
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--webserver_port=41185
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260806 07:50:08.664860 970 raft_consensus.cc:493] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e718584a9554459ea5b8a7cdc3fe1a2f)
I20260806 07:50:08.665153 970 raft_consensus.cc:515] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } }
I20260806 07:50:08.667402 970 leader_election.cc:290] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e718584a9554459ea5b8a7cdc3fe1a2f (127.20.202.254:45023), 528f43c399224da1be4b94f6f7ca0234 (127.20.202.253:37709)
I20260806 07:50:08.667775 32636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "681764fde77049e4944a3bc5f7dba68b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "528f43c399224da1be4b94f6f7ca0234" is_pre_election: true
I20260806 07:50:08.675276 768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "681764fde77049e4944a3bc5f7dba68b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" is_pre_election: true
I20260806 07:50:08.675683 768 raft_consensus.cc:2393] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 681764fde77049e4944a3bc5f7dba68b in current term 3: Already voted for candidate 528f43c399224da1be4b94f6f7ca0234 in this term.
I20260806 07:50:08.676362 32697 leader_election.cc:304] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 681764fde77049e4944a3bc5f7dba68b; no voters: 528f43c399224da1be4b94f6f7ca0234, e718584a9554459ea5b8a7cdc3fe1a2f
I20260806 07:50:08.676977 970 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:50:08.679147 970 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260806 07:50:08.951676 969 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:08.952015 969 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:08.952169 969 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:50:08.952247 969 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:08.961236 969 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:08.961496 969 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:08.967947 32721 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [term 3 FOLLOWER]: Refusing update from remote peer 528f43c399224da1be4b94f6f7ca0234: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:50:08.968842 964 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:50:08.971007 970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:08.971307 970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:08.973742 958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:08.974010 958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:08.974524 969 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41191
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41185
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:08.975035 768 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [term 3 FOLLOWER]: Refusing update from remote peer 528f43c399224da1be4b94f6f7ca0234: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:50:08.975859 970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:08.976114 970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681764fde77049e4944a3bc5f7dba68b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:08.976323 958 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:50:08.976779 969 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:08.979497 969 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:50:08.982885 982 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:08.983583 982 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:08.983973 979 sys_catalog.cc:455] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528f43c399224da1be4b94f6f7ca0234. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:08.984182 979 sys_catalog.cc:458] T 00000000000000000000000000000000 P e718584a9554459ea5b8a7cdc3fe1a2f [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:50:08.991539 985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:08.991580 984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:08.992667 987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:08.993789 969 server_base.cc:1061] running on GCE node
I20260806 07:50:08.994488 969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:08.995445 969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:08.996721 969 hybrid_clock.cc:648] HybridClock initialized: now 1786002608996655 us; error 57 us; skew 500 ppm
I20260806 07:50:08.997071 969 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:08.999573 969 webserver.cc:533] Webserver started at http://127.20.202.193:41185/ using document root <none> and password file <none>
I20260806 07:50:09.000553 969 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:09.000722 969 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:09.007339 969 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:50:09.010255 993 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:09.011668 969 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:09.011857 969 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:50:09.012626 969 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:09.029006 969 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:09.030097 969 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:09.030490 969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:09.031577 969 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:09.033387 969 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:09.033512 969 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:09.033699 969 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:09.033795 969 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:50:09.034238 969 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:50:09.034379 969 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:50:09.034477 969 fault_injection.cc:59] injecting fault for kudu (pid 969): FLAGS_fault_before_start (process will exit)
I20260806 07:50:09.060950 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41191
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--webserver_port=41185
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:50:09.405224 1001 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:09.405583 1001 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:09.405750 1001 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:09.414093 1001 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:09.414307 1001 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:09.424158 1001 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41191
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41185
--tserver_master_addrs=127.20.202.254:45023,127.20.202.253:37709,127.20.202.252:37635
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:09.426157 1001 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:09.428057 1001 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:09.438126 1006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:09.439146 1007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:09.439181 1009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:09.439646 1001 server_base.cc:1061] running on GCE node
I20260806 07:50:09.440075 978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "528f43c399224da1be4b94f6f7ca0234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e718584a9554459ea5b8a7cdc3fe1a2f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45023 } } peers { permanent_uuid: "528f43c399224da1be4b94f6f7ca0234" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37709 } } peers { permanent_uuid: "681764fde77049e4944a3bc5f7dba68b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37635 } } }
I20260806 07:50:09.440292 1001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:09.440379 978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528f43c399224da1be4b94f6f7ca0234 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:09.441396 1001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:09.442632 1001 hybrid_clock.cc:648] HybridClock initialized: now 1786002609442582 us; error 66 us; skew 500 ppm
I20260806 07:50:09.443001 1001 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:09.445425 1001 webserver.cc:533] Webserver started at http://127.20.202.193:41185/ using document root <none> and password file <none>
I20260806 07:50:09.446261 1001 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:09.446368 1001 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:09.451898 1001 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:50:09.454772 1016 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:09.456151 1001 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:09.456326 1001 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "816b04090cd44a58991cae25a346c47d"
format_stamp: "Formatted at 2026-08-06 07:49:49 on dist-test-slave-g350"
I20260806 07:50:09.457065 1001 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:09.487812 1001 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:09.488703 1001 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:09.489001 1001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:09.489988 1001 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:09.491725 1001 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:09.491842 1001 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:09.491938 1001 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:09.491990 1001 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:09.528044 1001 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41191
I20260806 07:50:09.528134 1129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41191 every 8 connection(s)
I20260806 07:50:09.529727 1001 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:09.535388 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1001
I20260806 07:50:09.535794 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1001
I20260806 07:50:09.545730 581 heartbeater.cc:499] Master 127.20.202.253:37709 was elected leader, sending a full tablet report...
I20260806 07:50:09.564175 1132 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37709
I20260806 07:50:09.564621 1132 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:09.566201 1131 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45023
I20260806 07:50:09.566411 1131 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:09.567117 1132 heartbeater.cc:507] Master 127.20.202.253:37709 requested a full tablet report, sending...
I20260806 07:50:09.567140 1131 heartbeater.cc:507] Master 127.20.202.254:45023 requested a full tablet report, sending...
I20260806 07:50:09.569420 758 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:09.571033 32625 ts_manager.cc:194] Re-registered known tserver with Master: 816b04090cd44a58991cae25a346c47d (127.20.202.193:41191)
I20260806 07:50:09.571491 1125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:09.573225 32625 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:59455
I20260806 07:50:09.668610 1130 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37635
I20260806 07:50:09.669500 1130 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:09.670042 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1001
I20260806 07:50:09.685511 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 449
I20260806 07:50:09.694080 574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:09.781483 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 449
I20260806 07:50:09.800312 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 587
I20260806 07:50:09.808441 710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:09.895596 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 587
I20260806 07:50:09.913161 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 725
I20260806 07:50:09.919631 788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:10.029203 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 725
I20260806 07:50:10.053318 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32578
I20260806 07:50:10.054754 32656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:10.165978 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32578
I20260806 07:50:10.188583 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32663
I20260806 07:50:10.189889 32741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:10.298079 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32663
Shutting down hive metastore.
2026-08-06T07:50:10Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/4 (39072 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/5
2026-08-06T07:50:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:10Z Disabled control of system clock
2026-08-06T07:50:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:10Z Disabled control of system clock
2026-08-06T07:50:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:10Z Disabled control of system clock
2026-08-06T07:50:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:10Z Disabled control of system clock
2026-08-06T07:50:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:10Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:50:18: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:50:26.716180 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
W20260806 07:50:27.055521 1445 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:27.055869 1445 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:27.056023 1445 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:27.064441 1445 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:27.064539 1445 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:27.064628 1445 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:27.064704 1445 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:27.075042 1445 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:27.076742 1445 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:27.078694 1445 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:27.087134 1451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:27.087622 1450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:27.088092 1453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:27.088165 1445 server_base.cc:1061] running on GCE node
I20260806 07:50:27.089013 1445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:27.090721 1445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:27.091959 1445 hybrid_clock.cc:648] HybridClock initialized: now 1786002627091840 us; error 90 us; skew 500 ppm
I20260806 07:50:27.092309 1445 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:27.094575 1445 webserver.cc:533] Webserver started at http://127.20.202.254:41689/ using document root <none> and password file <none>
I20260806 07:50:27.095315 1445 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:27.095458 1445 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:27.095830 1445 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:27.098338 1445 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "e331606af72448db8fa1556690bdcc1f"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:27.099138 1445 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "e331606af72448db8fa1556690bdcc1f"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:27.104554 1445 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:50:27.108084 1459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:27.109462 1445 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:50:27.109658 1445 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e331606af72448db8fa1556690bdcc1f"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:27.109879 1445 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:27.131742 1445 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:27.132594 1445 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:27.132889 1445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:27.153846 1445 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34501
I20260806 07:50:27.153869 1512 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34501 every 8 connection(s)
I20260806 07:50:27.155359 1445 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:27.159343 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1445
I20260806 07:50:27.159516 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:50:27.159710 1513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:50:27.160746 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39497
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
I20260806 07:50:27.165560 1513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:50:27.187001 1513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:50:27.188164 1460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111)
W20260806 07:50:27.189888 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39497: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111)
I20260806 07:50:27.239276 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } attempt: 1
W20260806 07:50:27.241501 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39497: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111)
I20260806 07:50:27.309861 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } attempt: 2
W20260806 07:50:27.312011 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39497: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111)
I20260806 07:50:27.403295 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } attempt: 3
W20260806 07:50:27.405473 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39497: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111)
W20260806 07:50:27.503659 1515 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:27.503963 1515 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:27.504050 1515 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:27.512389 1515 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:27.512483 1515 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:27.512539 1515 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:27.512578 1515 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:27.522601 1515 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:27.524201 1515 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:27.525988 1515 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:27.534766 1523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:27.534736 1524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:27.534898 1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:27.536368 1515 server_base.cc:1061] running on GCE node
I20260806 07:50:27.537087 1515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:27.538767 1515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:27.539973 1515 hybrid_clock.cc:648] HybridClock initialized: now 1786002627539856 us; error 86 us; skew 500 ppm
I20260806 07:50:27.540289 1515 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:27.542454 1515 webserver.cc:533] Webserver started at http://127.20.202.253:46529/ using document root <none> and password file <none>
I20260806 07:50:27.543138 1515 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:27.543226 1515 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:27.543537 1515 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:27.545970 1515 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "ce5894cd69cc49c994ed89e39707de43"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:27.546703 1515 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "ce5894cd69cc49c994ed89e39707de43"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:27.548847 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } attempt: 4
W20260806 07:50:27.550882 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39497: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111)
I20260806 07:50:27.552158 1515 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:50:27.555540 1532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:27.556926 1515 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:50:27.557060 1515 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "ce5894cd69cc49c994ed89e39707de43"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:27.557228 1515 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:27.587594 1515 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:27.588433 1515 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:27.588680 1515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:27.609124 1515 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:39497
I20260806 07:50:27.609155 1586 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:39497 every 8 connection(s)
I20260806 07:50:27.610683 1515 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:50:27.615152 1587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:50:27.616309 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1515
I20260806 07:50:27.616506 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:50:27.617985 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35863
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
I20260806 07:50:27.623893 1587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:50:27.637840 1587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:50:27.648221 1587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:50:27.649261 1536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
W20260806 07:50:27.651010 1587 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:27.700394 1587 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 1
W20260806 07:50:27.702628 1587 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:27.770910 1587 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 2
W20260806 07:50:27.773151 1587 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:27.850170 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } attempt: 5
I20260806 07:50:27.857554 1513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:50:27.859687 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:27.864420 1587 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 3
W20260806 07:50:27.866204 1587 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:27.910974 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 1
W20260806 07:50:27.913357 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
W20260806 07:50:27.967371 1589 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:27.967694 1589 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:27.967819 1589 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:27.976321 1589 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:27.976418 1589 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:27.976536 1589 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:27.976615 1589 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:27.981673 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 2
W20260806 07:50:27.983938 1513 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:27.987584 1589 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:27.989250 1589 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:27.991145 1589 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:27.999368 1598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:27.999774 1597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:28.000595 1589 server_base.cc:1061] running on GCE node
W20260806 07:50:28.001268 1600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:28.002221 1589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:28.003961 1589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:28.005175 1589 hybrid_clock.cc:648] HybridClock initialized: now 1786002628005019 us; error 126 us; skew 500 ppm
I20260806 07:50:28.005734 1589 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:28.007922 1589 webserver.cc:533] Webserver started at http://127.20.202.252:45615/ using document root <none> and password file <none>
I20260806 07:50:28.008657 1589 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:28.008749 1589 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:28.009116 1589 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:28.009429 1587 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 4
W20260806 07:50:28.011523 1587 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111)
I20260806 07:50:28.013484 1589 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "64879c8ec3a54c119c519b91fb363693"
format_stamp: "Formatted at 2026-08-06 07:50:28 on dist-test-slave-g350"
I20260806 07:50:28.014401 1589 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "64879c8ec3a54c119c519b91fb363693"
format_stamp: "Formatted at 2026-08-06 07:50:28 on dist-test-slave-g350"
I20260806 07:50:28.020036 1589 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:50:28.023411 1606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:28.024791 1589 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:50:28.024931 1589 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "64879c8ec3a54c119c519b91fb363693"
format_stamp: "Formatted at 2026-08-06 07:50:28 on dist-test-slave-g350"
I20260806 07:50:28.025101 1589 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:28.051735 1589 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:28.052574 1589 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:28.052809 1589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:28.074566 1589 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:35863
I20260806 07:50:28.074604 1659 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:35863 every 8 connection(s)
I20260806 07:50:28.076203 1589 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:50:28.076453 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1589
I20260806 07:50:28.076678 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:50:28.082618 1660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:50:28.090210 1513 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 3
I20260806 07:50:28.092156 1660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:50:28.109135 1513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap starting.
I20260806 07:50:28.109771 1660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:50:28.113469 1513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Neither blocks nor log segments found. Creating new log.
I20260806 07:50:28.115301 1513 log.cc:826] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:28.117020 1660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:50:28.118959 1513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: No bootstrap required, opened a new log
I20260806 07:50:28.127660 1513 raft_consensus.cc:359] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.128017 1513 raft_consensus.cc:385] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:50:28.128166 1513 raft_consensus.cc:740] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e331606af72448db8fa1556690bdcc1f, State: Initialized, Role: FOLLOWER
I20260806 07:50:28.129040 1660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Bootstrap starting.
I20260806 07:50:28.128934 1513 consensus_queue.cc:260] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.130686 1667 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.131023 1667 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.132011 1513 sys_catalog.cc:565] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:28.133496 1660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Neither blocks nor log segments found. Creating new log.
I20260806 07:50:28.135092 1660 log.cc:826] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Log is configured to *not* fsync() on all Append() calls
W20260806 07:50:28.135464 1465 tablet.cc:2406] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:50:28.138214 1660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: No bootstrap required, opened a new log
W20260806 07:50:28.144342 1679 catalog_manager.cc:1594] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:50:28.144577 1679 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:50:28.145989 1660 raft_consensus.cc:359] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.146507 1660 raft_consensus.cc:385] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:50:28.146689 1660 raft_consensus.cc:740] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64879c8ec3a54c119c519b91fb363693, State: Initialized, Role: FOLLOWER
I20260806 07:50:28.147535 1660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.148594 1681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.148932 1681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.149811 1660 sys_catalog.cc:565] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:28.143412 1445 master_runner.cc:187] Error getting master registration for 127.20.202.253:39497: OK, instance_id { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" instance_seqno: 1786002627603547 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:50:28.151299 1445 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:28.165720 1589 master_runner.cc:187] Error getting master registration for 127.20.202.253:39497: OK, instance_id { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" instance_seqno: 1786002627603547 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:50:28.173193 1589 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:28.173475 1692 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:50:28.173594 1692 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:50:28.305222 1612 tablet.cc:2406] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:50:28.310768 1587 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } attempt: 5
I20260806 07:50:28.324296 1587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Bootstrap starting.
I20260806 07:50:28.328719 1587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Neither blocks nor log segments found. Creating new log.
I20260806 07:50:28.330235 1587 log.cc:826] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:28.333853 1587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: No bootstrap required, opened a new log
I20260806 07:50:28.341845 1587 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.342204 1587 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:50:28.342332 1587 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5894cd69cc49c994ed89e39707de43, State: Initialized, Role: FOLLOWER
I20260806 07:50:28.343088 1587 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.358850 1695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.360209 1587 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:28.361932 1695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.378980 1515 master_runner.cc:264] No leader master found from master ce5894cd69cc49c994ed89e39707de43
I20260806 07:50:28.379129 1515 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:28.388638 1706 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:50:28.388787 1706 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:50:28.591616 1538 tablet.cc:2406] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:50:28.603628 1667 raft_consensus.cc:493] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:50:28.604269 1667 raft_consensus.cc:515] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.606220 1667 leader_election.cc:290] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497), 64879c8ec3a54c119c519b91fb363693 (127.20.202.252:35863)
I20260806 07:50:28.607811 1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64879c8ec3a54c119c519b91fb363693" is_pre_election: true
I20260806 07:50:28.608300 1633 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 0.
I20260806 07:50:28.609292 1463 leader_election.cc:304] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f; no voters:
I20260806 07:50:28.611017 1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43" is_pre_election: true
I20260806 07:50:28.611559 1560 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 0.
I20260806 07:50:28.612625 1667 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:50:28.612779 1667 raft_consensus.cc:493] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:50:28.612877 1667 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:50:28.617489 1667 raft_consensus.cc:515] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.618856 1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43"
I20260806 07:50:28.619074 1560 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:50:28.622769 1560 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 1.
I20260806 07:50:28.623335 1460 leader_election.cc:304] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce5894cd69cc49c994ed89e39707de43, e331606af72448db8fa1556690bdcc1f; no voters:
I20260806 07:50:28.626341 1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64879c8ec3a54c119c519b91fb363693"
I20260806 07:50:28.626570 1633 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:50:28.631829 1633 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 1.
I20260806 07:50:28.632727 1667 leader_election.cc:290] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 1 election: Requested vote from peers ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497), 64879c8ec3a54c119c519b91fb363693 (127.20.202.252:35863)
I20260806 07:50:28.633162 1667 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:50:28.633735 1667 raft_consensus.cc:697] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 LEADER]: Becoming Leader. State: Replica: e331606af72448db8fa1556690bdcc1f, State: Running, Role: LEADER
I20260806 07:50:28.634675 1667 consensus_queue.cc:237] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:28.646736 1708 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e331606af72448db8fa1556690bdcc1f. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.647082 1708 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:28.654106 1710 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:28.656284 1710 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:28.664587 1560 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 1 FOLLOWER]: Refusing update from remote peer e331606af72448db8fa1556690bdcc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:50:28.664589 1633 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Refusing update from remote peer e331606af72448db8fa1556690bdcc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:50:28.666751 1667 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:50:28.667465 1667 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:50:28.689815 1695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e331606af72448db8fa1556690bdcc1f. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.690222 1695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.695139 1713 mvcc.cc:204] Tried to move back new op lower bound from 7315466766998233088 to 7315466766904205312. Current Snapshot: MvccSnapshot[applied={T|T < 7315466766998233088}]
I20260806 07:50:28.701113 1712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e331606af72448db8fa1556690bdcc1f. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.701510 1712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.703740 1710 catalog_manager.cc:1383] Generated new cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:28.703881 1710 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:28.705670 1712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.705958 1712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.714921 1708 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.715165 1708 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:28.714897 1719 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.715523 1719 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:28.715833 1695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:28.716053 1695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:28.717703 1715 mvcc.cc:204] Tried to move back new op lower bound from 7315466766998233088 to 7315466766904205312. Current Snapshot: MvccSnapshot[applied={T|T < 7315466766998233088}]
I20260806 07:50:28.732861 1710 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:50:28.734843 1710 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:28.749993 1710 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Generated new TSK 0
I20260806 07:50:28.751161 1710 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:50:28.788514 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:50:29.134123 1721 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:29.134408 1721 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:29.134485 1721 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:29.143222 1721 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:29.143373 1721 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:29.153430 1721 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:29.155510 1721 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:29.157902 1721 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:29.169049 1726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:29.169955 1727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:29.170765 1729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:29.171300 1721 server_base.cc:1061] running on GCE node
I20260806 07:50:29.171871 1721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:29.173002 1721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:29.174342 1721 hybrid_clock.cc:648] HybridClock initialized: now 1786002629174208 us; error 89 us; skew 500 ppm
I20260806 07:50:29.174729 1721 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:29.175951 1692 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:29.176055 1692 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: loading cluster ID for follower catalog manager: success
I20260806 07:50:29.177641 1721 webserver.cc:533] Webserver started at http://127.20.202.193:43239/ using document root <none> and password file <none>
I20260806 07:50:29.178635 1721 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:29.178763 1721 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:29.179169 1721 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:29.179940 1692 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: acquiring CA information for follower catalog manager: success
I20260806 07:50:29.182319 1692 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:29.182524 1721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:29.183306 1721 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:29.188735 1721 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 07:50:29.191985 1736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:29.193352 1721 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:50:29.193490 1721 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:29.193735 1721 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:29.235839 1721 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:29.236701 1721 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:29.236936 1721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:29.237891 1721 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:29.239528 1721 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:29.239620 1721 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:29.239699 1721 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:29.239753 1721 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:29.275902 1721 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37447
I20260806 07:50:29.276010 1849 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37447 every 8 connection(s)
I20260806 07:50:29.277406 1721 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:29.282608 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1721
I20260806 07:50:29.282845 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:50:29.301013 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:29.317821 1852 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:29.317966 1850 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:29.318306 1850 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:29.318313 1852 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:29.319561 1850 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:29.320621 1852 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:29.321782 1851 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:29.321914 1851 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:29.322026 1477 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:37447)
I20260806 07:50:29.322381 1851 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:29.322500 1549 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:37447)
I20260806 07:50:29.324142 1623 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:37447)
I20260806 07:50:29.324188 1477 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:56745
I20260806 07:50:29.390645 1706 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:29.390787 1706 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: loading cluster ID for follower catalog manager: success
I20260806 07:50:29.393355 1706 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: acquiring CA information for follower catalog manager: success
I20260806 07:50:29.395134 1706 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:50:29.655121 1860 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:29.655660 1860 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:29.655795 1860 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:29.664597 1860 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:29.664803 1860 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:50:29.675071 1860 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:29.676752 1860 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:29.678711 1860 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:29.688599 1866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:29.688623 1865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:29.689195 1868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:29.690256 1860 server_base.cc:1061] running on GCE node
I20260806 07:50:29.690840 1860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:29.691912 1860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:29.693171 1860 hybrid_clock.cc:648] HybridClock initialized: now 1786002629693023 us; error 116 us; skew 500 ppm
I20260806 07:50:29.693492 1860 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:29.695600 1860 webserver.cc:533] Webserver started at http://127.20.202.194:44065/ using document root <none> and password file <none>
I20260806 07:50:29.696417 1860 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:29.696558 1860 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:29.696914 1860 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:29.699419 1860 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "69d5bd227a224ca69de5c4fc4d14f801"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:29.700191 1860 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "69d5bd227a224ca69de5c4fc4d14f801"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:29.705713 1860 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 07:50:29.709141 1875 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:29.710549 1860 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:29.710731 1860 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "69d5bd227a224ca69de5c4fc4d14f801"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:29.710940 1860 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:29.743613 1860 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:29.744975 1860 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:29.745332 1860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:29.746384 1860 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:29.748080 1860 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:29.748210 1860 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:29.748322 1860 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:29.748430 1860 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:29.784461 1860 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:35197
I20260806 07:50:29.784583 1988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:35197 every 8 connection(s)
I20260806 07:50:29.786132 1860 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:50:29.795166 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1860
I20260806 07:50:29.795413 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:50:29.802156 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:29.804410 1989 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:29.804795 1989 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:29.805235 1991 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:29.805388 1991 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:29.805814 1989 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:29.806067 1991 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:29.807406 1477 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:35197)
I20260806 07:50:29.807598 1549 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:35197)
I20260806 07:50:29.808373 1477 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:59445
I20260806 07:50:29.809128 1990 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:29.809284 1990 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:29.809824 1990 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:29.810844 1623 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:35197)
W20260806 07:50:30.155102 1996 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:30.155508 1996 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:30.155669 1996 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:30.164150 1996 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:30.164357 1996 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:50:30.174520 1996 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:30.176265 1996 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:30.178193 1996 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:30.189057 2004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:30.189258 2001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:30.189078 2002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:30.195148 1996 server_base.cc:1061] running on GCE node
I20260806 07:50:30.195788 1996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:30.196986 1996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:30.200363 1996 hybrid_clock.cc:648] HybridClock initialized: now 1786002630200220 us; error 106 us; skew 500 ppm
I20260806 07:50:30.200700 1996 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:30.202968 1996 webserver.cc:533] Webserver started at http://127.20.202.195:40187/ using document root <none> and password file <none>
I20260806 07:50:30.203749 1996 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:30.203883 1996 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:30.204241 1996 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:30.206672 1996 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "ee92a19da9e447988ad49738ab8f486f"
format_stamp: "Formatted at 2026-08-06 07:50:30 on dist-test-slave-g350"
I20260806 07:50:30.207665 1996 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "ee92a19da9e447988ad49738ab8f486f"
format_stamp: "Formatted at 2026-08-06 07:50:30 on dist-test-slave-g350"
I20260806 07:50:30.213071 1996 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:50:30.216686 2010 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:30.218106 1996 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:30.218279 1996 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "ee92a19da9e447988ad49738ab8f486f"
format_stamp: "Formatted at 2026-08-06 07:50:30 on dist-test-slave-g350"
I20260806 07:50:30.218536 1996 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:30.241523 1996 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:30.242425 1996 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:30.242713 1996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:30.243717 1996 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:30.245406 1996 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:30.245532 1996 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:30.245702 1996 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:30.245795 1996 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:30.281455 1996 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:37613
I20260806 07:50:30.281515 2124 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:37613 every 8 connection(s)
I20260806 07:50:30.282882 1996 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:50:30.290158 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1996
I20260806 07:50:30.290406 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:50:30.305567 2127 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:30.305672 2125 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:30.305774 2126 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:30.306061 2126 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:30.306066 2127 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:30.306061 2125 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:30.307160 2127 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:30.307340 2125 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:30.307384 2126 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:30.308929 1477 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:37613)
I20260806 07:50:30.309015 1549 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:37613)
I20260806 07:50:30.309017 1623 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:37613)
I20260806 07:50:30.310019 1477 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:44491
I20260806 07:50:30.318418 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:30.326301 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:30.326512 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1445
I20260806 07:50:30.326752 1850 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
I20260806 07:50:30.327435 1508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:30.440534 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1445
I20260806 07:50:30.464372 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--webserver_interface=127.20.202.254
--webserver_port=41689
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
W20260806 07:50:30.808787 2133 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:30.809149 2133 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:30.809288 2133 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:50:30.811219 1989 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
W20260806 07:50:30.812736 1989 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:34501 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:34501: connect: Connection refused (error 111)
W20260806 07:50:30.818562 2133 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:30.818641 2133 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:30.818697 2133 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:30.818747 2133 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:30.829270 2133 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=41689
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:30.831045 2133 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:30.832865 2133 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:30.841056 2140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:30.841202 2139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:30.842252 2142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:30.842411 2133 server_base.cc:1061] running on GCE node
I20260806 07:50:30.843135 2133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:30.844633 2133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:30.845896 2133 hybrid_clock.cc:648] HybridClock initialized: now 1786002630845731 us; error 120 us; skew 500 ppm
I20260806 07:50:30.846340 2133 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:30.848605 2133 webserver.cc:533] Webserver started at http://127.20.202.254:41689/ using document root <none> and password file <none>
I20260806 07:50:30.849331 2133 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:30.849471 2133 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:30.854813 2133 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:50:30.858189 2148 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:30.859617 2133 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:50:30.859804 2133 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e331606af72448db8fa1556690bdcc1f"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:30.860536 2133 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:30.877229 2133 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:30.878151 2133 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:30.878454 2133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:30.899078 2133 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34501
I20260806 07:50:30.899112 2200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34501 every 8 connection(s)
I20260806 07:50:30.900684 2133 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:30.907459 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2133
I20260806 07:50:30.907785 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1721
I20260806 07:50:30.907856 2201 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:30.912547 2201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap starting.
I20260806 07:50:30.917732 1845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:30.931962 2201 log.cc:826] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:30.940759 2201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:30.941306 2201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap complete.
I20260806 07:50:30.948065 2201 raft_consensus.cc:359] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:30.948797 2201 raft_consensus.cc:740] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e331606af72448db8fa1556690bdcc1f, State: Initialized, Role: FOLLOWER
I20260806 07:50:30.949652 2201 consensus_queue.cc:260] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:30.950590 2208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:30.950887 2208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:30.951715 2201 sys_catalog.cc:565] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:30.970104 2219 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:30.972796 2219 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: loading cluster ID for follower catalog manager: success
I20260806 07:50:30.983222 2219 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: acquiring CA information for follower catalog manager: success
I20260806 07:50:30.986325 2219 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:31.016209 2133 master_runner.cc:264] No leader master found from master e331606af72448db8fa1556690bdcc1f
I20260806 07:50:31.016407 2133 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:31.032542 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1721
I20260806 07:50:31.051949 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37447
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--webserver_port=43239
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:31.311970 2125 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
I20260806 07:50:31.323194 2166 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ee92a19da9e447988ad49738ab8f486f" instance_seqno: 1786002630272609) as {username='slave'} at 127.20.202.195:38741; Asking this server to re-register.
I20260806 07:50:31.324185 2125 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:31.326129 2166 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:37613)
I20260806 07:50:31.414047 2227 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e331606af72448db8fa1556690bdcc1f)
I20260806 07:50:31.414294 2227 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:31.415912 2227 leader_election.cc:290] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501), ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497)
I20260806 07:50:31.417451 1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43" is_pre_election: true
I20260806 07:50:31.426813 2176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e331606af72448db8fa1556690bdcc1f" is_pre_election: true
I20260806 07:50:31.427510 2176 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64879c8ec3a54c119c519b91fb363693 in term 1.
I20260806 07:50:31.428171 1608 leader_election.cc:304] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f; no voters: ce5894cd69cc49c994ed89e39707de43
I20260806 07:50:31.428639 2227 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:50:31.428777 2227 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e331606af72448db8fa1556690bdcc1f)
I20260806 07:50:31.428860 2227 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:50:31.431473 2227 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:31.432108 2227 leader_election.cc:290] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 2 election: Requested vote from peers e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501), ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497)
I20260806 07:50:31.432569 2176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e331606af72448db8fa1556690bdcc1f"
I20260806 07:50:31.432662 1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43"
I20260806 07:50:31.432780 2176 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:50:31.436767 2176 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64879c8ec3a54c119c519b91fb363693 in term 2.
I20260806 07:50:31.437215 1608 leader_election.cc:304] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f; no voters: ce5894cd69cc49c994ed89e39707de43
I20260806 07:50:31.437513 2227 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:50:31.437981 2227 raft_consensus.cc:697] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 LEADER]: Becoming Leader. State: Replica: 64879c8ec3a54c119c519b91fb363693, State: Running, Role: LEADER
I20260806 07:50:31.438423 2227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:31.440274 2232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64879c8ec3a54c119c519b91fb363693. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:31.440572 2232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:31.441293 2234 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:31.443068 2234 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:31.443850 2234 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:31.443953 2234 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:31.445056 2234 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:31.445904 2234 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Loaded TSK: 0
W20260806 07:50:31.446345 2225 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:31.446647 2225 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:31.446786 2225 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:50:31.446856 2234 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260806 07:50:31.455659 2225 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:31.455858 2225 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:31.466377 2225 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37447
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43239
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:31.468020 2225 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:31.469923 2225 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:31.479957 2236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:31.479990 2237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:31.480935 2239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:31.481905 2225 server_base.cc:1061] running on GCE node
I20260806 07:50:31.482539 2225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:31.483598 2225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:31.484831 2225 hybrid_clock.cc:648] HybridClock initialized: now 1786002631484709 us; error 90 us; skew 500 ppm
I20260806 07:50:31.485183 2225 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:31.487363 2225 webserver.cc:533] Webserver started at http://127.20.202.193:43239/ using document root <none> and password file <none>
I20260806 07:50:31.488132 2225 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:31.488292 2225 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:31.493727 2225 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:50:31.496330 2245 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:31.497726 2225 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:31.497939 2225 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:31.498651 2225 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:31.520514 2225 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:31.521426 2225 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:31.521785 2225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:31.522763 2225 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:31.524608 2225 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:31.524713 2225 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:31.524869 2225 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:31.524955 2225 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:31.562121 2225 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37447
I20260806 07:50:31.562153 2359 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37447 every 8 connection(s)
I20260806 07:50:31.563908 2225 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:31.571949 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2225
I20260806 07:50:31.598181 2362 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:31.598493 2360 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:31.598641 2360 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:31.598639 2362 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:31.599882 2362 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:31.599910 2360 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:31.601878 2166 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:37447)
I20260806 07:50:31.602213 1549 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:37447)
I20260806 07:50:31.617020 2361 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:31.617194 2361 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:31.617769 2361 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:31.619117 1623 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:37447)
I20260806 07:50:31.621356 1623 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:36575
I20260806 07:50:31.624130 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2225
I20260806 07:50:31.630909 2371 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e331606af72448db8fa1556690bdcc1f)
I20260806 07:50:31.631246 2371 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:31.632896 2371 leader_election.cc:290] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501), 64879c8ec3a54c119c519b91fb363693 (127.20.202.252:35863)
I20260806 07:50:31.634075 1633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce5894cd69cc49c994ed89e39707de43" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64879c8ec3a54c119c519b91fb363693" is_pre_election: true
I20260806 07:50:31.640306 2354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:31.640942 2176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce5894cd69cc49c994ed89e39707de43" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e331606af72448db8fa1556690bdcc1f" is_pre_election: true
I20260806 07:50:31.641469 2176 raft_consensus.cc:2393] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce5894cd69cc49c994ed89e39707de43 in current term 2: Already voted for candidate 64879c8ec3a54c119c519b91fb363693 in this term.
I20260806 07:50:31.642421 1534 leader_election.cc:304] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce5894cd69cc49c994ed89e39707de43; no voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f
I20260806 07:50:31.642891 2371 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:50:31.645839 2371 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:50:31.728051 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2225
I20260806 07:50:31.746752 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37447
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--webserver_port=43239
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260806 07:50:31.821738 1989 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:31.821972 1989 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
I20260806 07:50:31.822990 2166 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "69d5bd227a224ca69de5c4fc4d14f801" instance_seqno: 1786002629776080) as {username='slave'} at 127.20.202.194:48503; Asking this server to re-register.
I20260806 07:50:31.823846 1989 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:31.825178 2166 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:35197)
I20260806 07:50:32.000521 2176 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Refusing update from remote peer 64879c8ec3a54c119c519b91fb363693: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:50:32.001413 2378 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:50:32.005509 2379 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64879c8ec3a54c119c519b91fb363693. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:32.005868 2379 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:32.007376 2378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:32.007694 2378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:32.009186 1560 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 2 FOLLOWER]: Refusing update from remote peer 64879c8ec3a54c119c519b91fb363693: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:50:32.016134 2378 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:50:32.021123 2379 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:32.021360 2379 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:32.022034 2371 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64879c8ec3a54c119c519b91fb363693. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:32.022148 2388 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:32.022353 2371 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:32.022488 2388 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:50:32.109529 2376 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:32.109907 2376 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:32.110044 2376 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:50:32.110155 2376 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:32.119099 2376 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:32.119343 2376 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:32.129449 2376 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37447
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43239
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:32.131580 2376 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:32.133437 2376 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:32.143092 2393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:32.143481 2394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:32.143996 2396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:32.145262 2376 server_base.cc:1061] running on GCE node
I20260806 07:50:32.145941 2376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:32.147053 2376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:32.148355 2376 hybrid_clock.cc:648] HybridClock initialized: now 1786002632148148 us; error 124 us; skew 500 ppm
I20260806 07:50:32.148662 2376 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:32.151027 2376 webserver.cc:533] Webserver started at http://127.20.202.193:43239/ using document root <none> and password file <none>
I20260806 07:50:32.151785 2376 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:32.151939 2376 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:32.157733 2376 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:50:32.160459 2402 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:32.161927 2376 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:32.162108 2376 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:32.162851 2376 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:32.201493 2376 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:32.202469 2376 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:32.202762 2376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:32.203807 2376 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:32.205603 2376 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:32.205722 2376 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 07:50:32.205852 2376 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:32.205941 2376 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:50:32.206341 2376 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:50:32.206457 2376 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:50:32.206537 2376 fault_injection.cc:59] injecting fault for kudu (pid 2376): FLAGS_fault_before_start (process will exit)
I20260806 07:50:32.231027 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37447
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--webserver_port=43239
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:50:32.572960 2410 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:32.573316 2410 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:32.573462 2410 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:32.581811 2410 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:32.582044 2410 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:32.591952 2410 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42561,127.20.202.213:35057,127.20.202.214:44581,127.20.202.215:37793,127.20.202.216:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37447
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43239
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:32.593745 2410 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:32.595634 2410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:32.605497 2415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:32.606041 2416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:32.606401 2418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:32.607601 2410 server_base.cc:1061] running on GCE node
I20260806 07:50:32.608225 2410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:32.609361 2410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:32.610749 2410 hybrid_clock.cc:648] HybridClock initialized: now 1786002632610544 us; error 165 us; skew 500 ppm
I20260806 07:50:32.611089 2410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:32.613341 2410 webserver.cc:533] Webserver started at http://127.20.202.193:43239/ using document root <none> and password file <none>
I20260806 07:50:32.614198 2410 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:32.614313 2410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:32.620093 2410 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:50:32.623149 2424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:32.624626 2410 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:32.624801 2410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:32.625560 2410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:32.626471 2378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:32.626824 2378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:32.672642 2410 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:32.673555 2410 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:32.673923 2410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:32.674995 2410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:32.676673 2410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:32.676748 2410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:32.676885 2410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:32.676941 2410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:32.713243 2410 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37447
I20260806 07:50:32.713275 2541 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37447 every 8 connection(s)
I20260806 07:50:32.714892 2410 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:32.715372 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2410
I20260806 07:50:32.715778 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2410
I20260806 07:50:32.756064 2542 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:32.756539 2543 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:32.756651 2542 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:32.756762 2543 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:32.756917 2537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:32.815281 1990 heartbeater.cc:499] Master 127.20.202.252:35863 was elected leader, sending a full tablet report...
I20260806 07:50:32.843712 2544 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:32.844002 2544 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:32.844738 2544 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:32.845151 2543 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:32.845182 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2410
I20260806 07:50:32.864084 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1860
I20260806 07:50:32.872243 1984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:32.959686 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1860
I20260806 07:50:32.977753 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1996
I20260806 07:50:32.985970 2120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:33.079608 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1996
I20260806 07:50:33.102267 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2133
I20260806 07:50:33.110644 2196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:33.216454 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2133
W20260806 07:50:33.238207 1608 connection.cc:573] client connection to 127.20.202.254:34501 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:50:33.238525 1589 master_runner.cc:217] Error getting master consensus for 127.20.202.254:34501: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:50:33.238566 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1515
I20260806 07:50:33.238780 1589 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:33.239981 1581 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:50:33.335364 1608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:34501: connect: Connection refused (error 111)
W20260806 07:50:33.345824 1608 consensus_peers.cc:597] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 -> Peer e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501): Couldn't send request to peer e331606af72448db8fa1556690bdcc1f. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:34501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:50:33.349094 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1515
I20260806 07:50:33.371738 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1589
I20260806 07:50:33.373114 1654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:33.484369 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1589
Shutting down hive metastore.
2026-08-06T07:50:33Z chronyd exiting
2026-08-06T07:50:33Z chronyd exiting
2026-08-06T07:50:33Z chronyd exiting
2026-08-06T07:50:33Z chronyd exiting
2026-08-06T07:50:33Z chronyd exiting
2026-08-06T07:50:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:33Z Disabled control of system clock
2026-08-06T07:50:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:33Z Disabled control of system clock
2026-08-06T07:50:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:34Z Disabled control of system clock
2026-08-06T07:50:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:34Z Disabled control of system clock
2026-08-06T07:50:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:50:34Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:50:34: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:50:43.223039 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
W20260806 07:50:43.559757 2756 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:43.560228 2756 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:43.560348 2756 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:43.568819 2756 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:43.568915 2756 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:43.568971 2756 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:43.569018 2756 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:43.579339 2756 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:43.581012 2756 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:43.582815 2756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:43.591310 2761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:43.591379 2762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:43.591545 2764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:43.592203 2756 server_base.cc:1061] running on GCE node
I20260806 07:50:43.593182 2756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:43.594992 2756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:43.596218 2756 hybrid_clock.cc:648] HybridClock initialized: now 1786002643596090 us; error 94 us; skew 500 ppm
I20260806 07:50:43.596623 2756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:43.599020 2756 webserver.cc:533] Webserver started at http://127.20.202.254:44813/ using document root <none> and password file <none>
I20260806 07:50:43.599830 2756 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:43.599982 2756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:43.605926 2756 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:50:43.610451 2770 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:43.611980 2756 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:50:43.612155 2756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e331606af72448db8fa1556690bdcc1f"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:43.612929 2756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:43.652500 2756 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:43.653465 2756 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:43.653808 2756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:43.675650 2756 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34501
I20260806 07:50:43.675683 2822 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34501 every 8 connection(s)
I20260806 07:50:43.677390 2756 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:43.684612 2823 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:43.687063 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2756
I20260806 07:50:43.687273 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:50:43.688501 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39497
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
I20260806 07:50:43.688506 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap starting.
I20260806 07:50:43.707862 2823 log.cc:826] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:43.717048 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:43.717723 2823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap complete.
I20260806 07:50:43.725181 2823 raft_consensus.cc:359] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:43.725930 2823 raft_consensus.cc:740] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e331606af72448db8fa1556690bdcc1f, State: Initialized, Role: FOLLOWER
I20260806 07:50:43.726609 2823 consensus_queue.cc:260] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:43.727531 2827 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:43.727860 2827 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:43.728442 2823 sys_catalog.cc:565] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:43.738929 2838 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:43.739085 2838 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: loading cluster ID for follower catalog manager: success
I20260806 07:50:43.743434 2838 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: acquiring CA information for follower catalog manager: success
I20260806 07:50:43.746112 2838 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:43.763023 2756 master_runner.cc:187] Error getting master registration for 127.20.202.253:39497: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39497: connect: Connection refused (error 111),
I20260806 07:50:43.769101 2756 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:44.047075 2825 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:44.047410 2825 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:44.047483 2825 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:44.056223 2825 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:44.056354 2825 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:44.056434 2825 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:44.056563 2825 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:44.066952 2825 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:44.068614 2825 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:44.070528 2825 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:44.079733 2846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:44.079772 2847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:44.081092 2849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:44.082274 2825 server_base.cc:1061] running on GCE node
I20260806 07:50:44.083405 2825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:44.085948 2825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:44.087275 2825 hybrid_clock.cc:648] HybridClock initialized: now 1786002644087075 us; error 157 us; skew 500 ppm
I20260806 07:50:44.087774 2825 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:44.091037 2825 webserver.cc:533] Webserver started at http://127.20.202.253:46717/ using document root <none> and password file <none>
I20260806 07:50:44.092176 2825 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:44.092355 2825 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:44.100346 2825 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 07:50:44.104404 2855 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:44.106354 2825 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:50:44.106556 2825 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "ce5894cd69cc49c994ed89e39707de43"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:44.107707 2825 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:44.134536 2825 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:44.135459 2825 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:44.135788 2825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:44.158146 2825 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:39497
I20260806 07:50:44.158200 2906 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:39497 every 8 connection(s)
I20260806 07:50:44.159922 2825 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:50:44.165316 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2825
I20260806 07:50:44.165561 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:50:44.166602 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35863
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
I20260806 07:50:44.169260 2907 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:44.174083 2907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Bootstrap starting.
I20260806 07:50:44.190799 2907 log.cc:826] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:44.199900 2907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:44.200459 2907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: Bootstrap complete.
I20260806 07:50:44.207168 2907 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:44.207870 2907 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5894cd69cc49c994ed89e39707de43, State: Initialized, Role: FOLLOWER
I20260806 07:50:44.208645 2907 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:44.209561 2911 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:44.209894 2911 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:44.210826 2907 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:44.221946 2922 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:44.222066 2922 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: loading cluster ID for follower catalog manager: success
I20260806 07:50:44.224903 2922 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: acquiring CA information for follower catalog manager: success
I20260806 07:50:44.226817 2922 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:44.250885 2825 master_runner.cc:187] Error getting master registration for 127.20.202.252:35863: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35863: connect: Connection refused (error 111),
I20260806 07:50:44.254693 2825 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:44.544167 2909 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:44.544483 2909 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:44.544602 2909 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:44.553138 2909 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:44.553238 2909 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:44.553321 2909 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:44.553416 2909 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:44.564230 2909 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:44.566077 2909 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:44.567988 2909 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:44.576843 2932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:44.577739 2931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:44.581192 2934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:44.581357 2909 server_base.cc:1061] running on GCE node
I20260806 07:50:44.582345 2909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:44.584125 2909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:44.587546 2909 hybrid_clock.cc:648] HybridClock initialized: now 1786002644587346 us; error 122 us; skew 500 ppm
I20260806 07:50:44.588009 2909 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:44.590699 2909 webserver.cc:533] Webserver started at http://127.20.202.252:46325/ using document root <none> and password file <none>
I20260806 07:50:44.591732 2909 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:44.591898 2909 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:44.598287 2909 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:50:44.601578 2940 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:44.603264 2909 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:44.603504 2909 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "64879c8ec3a54c119c519b91fb363693"
format_stamp: "Formatted at 2026-08-06 07:50:28 on dist-test-slave-g350"
I20260806 07:50:44.604338 2909 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:44.631145 2909 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:44.632083 2909 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:44.632408 2909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:44.653980 2991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:35863 every 8 connection(s)
I20260806 07:50:44.653987 2909 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:35863
I20260806 07:50:44.655799 2909 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:50:44.661686 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2909
I20260806 07:50:44.661885 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:50:44.663408 2992 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:44.667961 2992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Bootstrap starting.
I20260806 07:50:44.688764 2992 log.cc:826] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:44.698247 2992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:44.699105 2992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Bootstrap complete.
I20260806 07:50:44.705971 2992 raft_consensus.cc:359] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:44.706691 2992 raft_consensus.cc:740] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64879c8ec3a54c119c519b91fb363693, State: Initialized, Role: FOLLOWER
I20260806 07:50:44.707409 2992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:44.708225 2997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:44.708474 2997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:44.709458 2992 sys_catalog.cc:565] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:44.724802 3008 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:44.724939 3008 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: loading cluster ID for follower catalog manager: success
I20260806 07:50:44.728689 3008 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: acquiring CA information for follower catalog manager: success
I20260806 07:50:44.731385 3008 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:44.747231 2909 master_runner.cc:264] No leader master found from master 64879c8ec3a54c119c519b91fb363693
I20260806 07:50:44.747396 2909 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:45.086740 3011 raft_consensus.cc:493] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:50:45.087065 3011 raft_consensus.cc:515] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:45.089075 3011 leader_election.cc:290] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497), 64879c8ec3a54c119c519b91fb363693 (127.20.202.252:35863)
I20260806 07:50:45.098630 2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64879c8ec3a54c119c519b91fb363693" is_pre_election: true
I20260806 07:50:45.098834 2882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43" is_pre_election: true
I20260806 07:50:45.099344 2882 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 2.
I20260806 07:50:45.099479 2967 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 2.
I20260806 07:50:45.100116 2771 leader_election.cc:304] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce5894cd69cc49c994ed89e39707de43, e331606af72448db8fa1556690bdcc1f; no voters:
I20260806 07:50:45.100533 3011 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:50:45.100709 3011 raft_consensus.cc:493] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:50:45.100829 3011 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:50:45.104532 3011 raft_consensus.cc:515] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:45.105203 3011 leader_election.cc:290] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 3 election: Requested vote from peers ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497), 64879c8ec3a54c119c519b91fb363693 (127.20.202.252:35863)
I20260806 07:50:45.105717 2882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43"
I20260806 07:50:45.105790 2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e331606af72448db8fa1556690bdcc1f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64879c8ec3a54c119c519b91fb363693"
I20260806 07:50:45.105924 2882 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:50:45.105993 2967 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:50:45.109422 2882 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 3.
I20260806 07:50:45.110211 2771 leader_election.cc:304] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce5894cd69cc49c994ed89e39707de43, e331606af72448db8fa1556690bdcc1f; no voters:
I20260806 07:50:45.110580 3011 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:50:45.111109 3011 raft_consensus.cc:697] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 LEADER]: Becoming Leader. State: Replica: e331606af72448db8fa1556690bdcc1f, State: Running, Role: LEADER
I20260806 07:50:45.111706 3011 consensus_queue.cc:237] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:45.112233 2967 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e331606af72448db8fa1556690bdcc1f in term 3.
I20260806 07:50:45.114935 3014 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e331606af72448db8fa1556690bdcc1f. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.115311 3014 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:45.116122 3016 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:45.118153 3016 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:45.118965 3016 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:45.119059 3016 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:45.120163 3016 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:45.121001 3016 catalog_manager.cc:6081] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Loaded TSK: 0
I20260806 07:50:45.121938 3016 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:50:45.133350 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:50:45.492947 3017 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:45.493288 3017 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:45.493429 3017 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:45.501868 3017 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:45.502110 3017 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:45.512336 3017 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:45.514178 3017 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:45.516237 3017 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:50:45.524650 2882 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 3 FOLLOWER]: Refusing update from remote peer e331606af72448db8fa1556690bdcc1f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:50:45.526240 3014 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:50:45.526563 3022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:45.527021 3023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:45.528544 3017 server_base.cc:1061] running on GCE node
W20260806 07:50:45.528967 3025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:45.529803 3017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:45.530920 3027 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e331606af72448db8fa1556690bdcc1f. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.531268 3027 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:45.531323 3017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:45.532917 3014 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.533202 3014 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:45.534201 2967 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Refusing update from remote peer e331606af72448db8fa1556690bdcc1f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:50:45.534969 3014 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:50:45.539126 3017 hybrid_clock.cc:648] HybridClock initialized: now 1786002645538914 us; error 144 us; skew 500 ppm
I20260806 07:50:45.539736 3017 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:45.545922 3027 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.546195 3027 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:45.546166 3017 webserver.cc:533] Webserver started at http://127.20.202.193:34293/ using document root <none> and password file <none>
I20260806 07:50:45.547339 3017 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:45.547556 3017 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:45.547782 3038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.548031 3038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:45.548712 3033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e331606af72448db8fa1556690bdcc1f. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.548961 3033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:45.553993 3017 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:50:45.556865 3040 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:45.558435 3017 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:45.558624 3017 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:45.559623 3017 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:45.585986 3017 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:45.586802 3017 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:45.587056 3017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:45.587963 3017 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:45.589651 3017 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:45.589723 3017 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:50:45.589807 3017 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:45.589849 3017 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:45.626122 3017 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42663
I20260806 07:50:45.626195 3155 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42663 every 8 connection(s)
I20260806 07:50:45.627770 3017 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:45.629583 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3017
I20260806 07:50:45.629895 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:50:45.643963 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:45.666970 3156 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:45.667326 3156 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:45.668488 3156 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:45.671072 2787 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:45.672645 2787 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:41229
I20260806 07:50:45.673944 3157 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:45.674089 3157 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:45.674645 3157 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:45.675037 3158 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:45.675170 3158 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:45.675702 3158 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:45.677660 2872 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:45.679091 2957 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:45.952996 3031 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e331606af72448db8fa1556690bdcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:45.953359 3031 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: LEADER
W20260806 07:50:46.010394 3166 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:46.010730 3166 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:46.010864 3166 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:46.019832 3166 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:46.020054 3166 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:50:46.030150 3166 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:46.031838 3166 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:46.033800 3166 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:46.043602 3171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:46.043769 3172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:46.044716 3174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:46.045061 3166 server_base.cc:1061] running on GCE node
I20260806 07:50:46.045805 3166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:46.047047 3166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:46.048331 3166 hybrid_clock.cc:648] HybridClock initialized: now 1786002646048154 us; error 118 us; skew 500 ppm
I20260806 07:50:46.048679 3166 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:46.050992 3166 webserver.cc:533] Webserver started at http://127.20.202.194:35829/ using document root <none> and password file <none>
I20260806 07:50:46.051733 3166 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:46.051839 3166 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:46.057458 3166 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:50:46.060714 3180 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:46.062309 3166 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:46.062520 3166 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "69d5bd227a224ca69de5c4fc4d14f801"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:46.063243 3166 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:46.100409 3166 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:46.101321 3166 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:46.101691 3166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:46.102762 3166 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:46.104560 3166 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:46.104633 3166 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:46.104784 3166 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:46.104841 3166 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:46.141994 3166 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:41745
I20260806 07:50:46.142056 3294 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:41745 every 8 connection(s)
I20260806 07:50:46.143512 3166 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:50:46.148224 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3166
I20260806 07:50:46.148455 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:50:46.153525 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:46.175105 3297 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:46.175206 3296 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:46.175693 3297 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:46.175694 3296 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:46.176987 3297 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:46.177009 3296 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:46.179159 2957 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:41745)
I20260806 07:50:46.179165 2872 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:41745)
I20260806 07:50:46.185937 3295 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:46.186093 3295 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:46.186578 3295 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:46.187649 2787 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:41745)
I20260806 07:50:46.188845 2787 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:44729
W20260806 07:50:46.504357 3302 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:46.504664 3302 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:46.504740 3302 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:46.513259 3302 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:46.513429 3302 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:50:46.523752 3302 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:46.525517 3302 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:46.527462 3302 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:46.538338 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:46.538467 3310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:46.538296 3308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:46.539472 3302 server_base.cc:1061] running on GCE node
I20260806 07:50:46.540100 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:46.541260 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:46.542641 3302 hybrid_clock.cc:648] HybridClock initialized: now 1786002646542460 us; error 146 us; skew 500 ppm
I20260806 07:50:46.542975 3302 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:46.545220 3302 webserver.cc:533] Webserver started at http://127.20.202.195:37573/ using document root <none> and password file <none>
I20260806 07:50:46.546089 3302 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:46.546241 3302 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:46.551790 3302 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:50:46.554790 3316 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:46.556327 3302 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:50:46.556470 3302 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "ee92a19da9e447988ad49738ab8f486f"
format_stamp: "Formatted at 2026-08-06 07:50:30 on dist-test-slave-g350"
I20260806 07:50:46.557205 3302 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:46.581694 3302 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:46.582681 3302 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:46.582984 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:46.584002 3302 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:46.585836 3302 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:46.585911 3302 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:46.586102 3302 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:46.586158 3302 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:46.622440 3302 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:43743
I20260806 07:50:46.622534 3429 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:43743 every 8 connection(s)
I20260806 07:50:46.624024 3302 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:50:46.625238 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3302
I20260806 07:50:46.625460 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:50:46.651525 3431 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:46.651844 3432 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:46.652043 3432 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:46.652756 3430 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:46.652921 3430 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:46.653309 3432 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:46.653455 3430 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:46.652040 3431 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:46.654640 3431 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:46.655200 2787 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:43743)
I20260806 07:50:46.655372 2871 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:43743)
I20260806 07:50:46.656270 2787 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:36765
I20260806 07:50:46.657065 2957 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:43743)
I20260806 07:50:46.672089 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:46.678061 3156 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
I20260806 07:50:46.682178 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:46.682356 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2756
I20260806 07:50:46.683284 2818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:46.792800 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2756
I20260806 07:50:46.817184 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--webserver_interface=127.20.202.254
--webserver_port=44813
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863 with env {}
W20260806 07:50:47.176368 3438 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:47.176697 3438 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:47.176868 3438 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:47.186192 3438 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:47.186285 3438 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:47.186374 3438 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:47.186471 3438 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:47.191588 3295 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
W20260806 07:50:47.193292 3295 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:34501 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:34501: connect: Connection refused (error 111)
I20260806 07:50:47.196944 3438 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=44813
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:47.198678 3438 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:47.200577 3438 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:47.208593 3444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:47.208781 3445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:47.209312 3447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:47.210325 3438 server_base.cc:1061] running on GCE node
I20260806 07:50:47.211138 3438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:47.212947 3438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:47.214175 3438 hybrid_clock.cc:648] HybridClock initialized: now 1786002647214053 us; error 93 us; skew 500 ppm
I20260806 07:50:47.214540 3438 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:47.217023 3438 webserver.cc:533] Webserver started at http://127.20.202.254:44813/ using document root <none> and password file <none>
I20260806 07:50:47.217839 3438 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:47.217991 3438 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:47.223377 3438 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:50:47.226251 3453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:47.227600 3438 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:47.227779 3438 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "e331606af72448db8fa1556690bdcc1f"
format_stamp: "Formatted at 2026-08-06 07:50:27 on dist-test-slave-g350"
I20260806 07:50:47.228497 3438 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:47.251641 3438 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:47.252560 3438 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:47.252885 3438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:47.274458 3438 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34501
I20260806 07:50:47.274482 3504 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34501 every 8 connection(s)
I20260806 07:50:47.276269 3438 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:47.282226 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3438
I20260806 07:50:47.282572 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3017
I20260806 07:50:47.284462 3505 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:47.287941 3505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap starting.
I20260806 07:50:47.292393 3151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:47.306932 3505 log.cc:826] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:47.317379 3505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:47.318081 3505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: Bootstrap complete.
I20260806 07:50:47.324868 3505 raft_consensus.cc:359] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:47.325659 3505 raft_consensus.cc:740] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e331606af72448db8fa1556690bdcc1f, State: Initialized, Role: FOLLOWER
I20260806 07:50:47.326525 3505 consensus_queue.cc:260] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:47.327433 3511 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:47.327734 3511 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:47.328579 3505 sys_catalog.cc:565] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:47.339203 3522 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:47.339411 3522 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: loading cluster ID for follower catalog manager: success
I20260806 07:50:47.343991 3522 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: acquiring CA information for follower catalog manager: success
I20260806 07:50:47.346776 3522 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:47.383136 3438 master_runner.cc:264] No leader master found from master e331606af72448db8fa1556690bdcc1f
I20260806 07:50:47.383322 3438 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:47.393400 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3017
I20260806 07:50:47.413196 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42663
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--webserver_port=34293
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:47.659219 3430 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
I20260806 07:50:47.681548 3470 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ee92a19da9e447988ad49738ab8f486f" instance_seqno: 1786002646613399) as {username='slave'} at 127.20.202.195:48967; Asking this server to re-register.
I20260806 07:50:47.682626 3430 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:47.684582 3470 ts_manager.cc:194] Registered new tserver with Master: ee92a19da9e447988ad49738ab8f486f (127.20.202.195:43743)
W20260806 07:50:47.773002 3528 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:47.773329 3528 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:47.773466 3528 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:47.781862 3528 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:47.782081 3528 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:47.792189 3528 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42663
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=34293
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:47.793855 3528 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:47.795691 3528 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:47.805696 3534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:47.805738 3535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:47.807502 3537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:47.807667 3528 server_base.cc:1061] running on GCE node
I20260806 07:50:47.808312 3528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:47.809558 3528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:47.810896 3528 hybrid_clock.cc:648] HybridClock initialized: now 1786002647810751 us; error 113 us; skew 500 ppm
I20260806 07:50:47.811264 3528 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:47.813524 3528 webserver.cc:533] Webserver started at http://127.20.202.193:34293/ using document root <none> and password file <none>
I20260806 07:50:47.814404 3528 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:47.814540 3528 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:47.820135 3528 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:50:47.822883 3543 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:47.824275 3528 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:47.824451 3528 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:47.825181 3528 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:47.862272 3528 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:47.863252 3528 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:47.863550 3528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:47.864535 3528 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:47.866302 3528 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:47.866402 3528 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:47.866551 3528 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:47.866621 3528 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:47.901677 3528 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42663
I20260806 07:50:47.901742 3656 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42663 every 8 connection(s)
I20260806 07:50:47.903224 3528 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:47.906831 3660 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e331606af72448db8fa1556690bdcc1f)
I20260806 07:50:47.907157 3660 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:47.908754 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3528
I20260806 07:50:47.924450 2882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43" is_pre_election: true
I20260806 07:50:47.925935 3660 leader_election.cc:290] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501), ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497)
I20260806 07:50:47.941781 3480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e331606af72448db8fa1556690bdcc1f" is_pre_election: true
I20260806 07:50:47.942396 3480 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64879c8ec3a54c119c519b91fb363693 in term 3.
I20260806 07:50:47.943109 2942 leader_election.cc:304] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f; no voters: ce5894cd69cc49c994ed89e39707de43
I20260806 07:50:47.943532 3660 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 07:50:47.943660 3660 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e331606af72448db8fa1556690bdcc1f)
I20260806 07:50:47.943758 3660 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:50:47.947266 3660 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:47.948518 3480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e331606af72448db8fa1556690bdcc1f"
I20260806 07:50:47.948724 3480 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:50:47.949692 3657 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:47.950093 3657 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:47.951117 3657 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:47.956606 3480 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64879c8ec3a54c119c519b91fb363693 in term 4.
I20260806 07:50:47.958063 3470 ts_manager.cc:194] Registered new tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:47.958320 3660 leader_election.cc:290] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 4 election: Requested vote from peers e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501), ce5894cd69cc49c994ed89e39707de43 (127.20.202.253:39497)
I20260806 07:50:47.959074 2882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64879c8ec3a54c119c519b91fb363693" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce5894cd69cc49c994ed89e39707de43"
I20260806 07:50:47.959477 3659 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:47.959627 3659 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:47.960186 3659 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:47.960638 2942 leader_election.cc:304] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f; no voters: ce5894cd69cc49c994ed89e39707de43
I20260806 07:50:47.961390 3660 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 4 FOLLOWER]: Leader election won for term 4
I20260806 07:50:47.962137 2871 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:47.965876 3660 raft_consensus.cc:697] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [term 4 LEADER]: Becoming Leader. State: Replica: 64879c8ec3a54c119c519b91fb363693, State: Running, Role: LEADER
I20260806 07:50:47.966485 3660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:47.968545 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64879c8ec3a54c119c519b91fb363693. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:47.968827 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:47.970696 3658 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:47.970851 3658 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:47.971113 3672 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:47.971361 3658 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:47.972601 2957 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:47.974064 3672 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:47.975097 3672 catalog_manager.cc:1295] Loaded cluster ID: ce1cf54d6a0549048e10c2bfad71a0c3
I20260806 07:50:47.975186 3672 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:47.976702 3672 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:47.977768 3672 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693: Loaded TSK: 0
I20260806 07:50:47.978941 3672 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:50:48.059623 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3528
I20260806 07:50:48.075656 3652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:48.131784 3676 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e331606af72448db8fa1556690bdcc1f)
I20260806 07:50:48.132056 3676 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } }
I20260806 07:50:48.134377 3676 leader_election.cc:290] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501), 64879c8ec3a54c119c519b91fb363693 (127.20.202.252:35863)
I20260806 07:50:48.155879 3480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce5894cd69cc49c994ed89e39707de43" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e331606af72448db8fa1556690bdcc1f" is_pre_election: true
I20260806 07:50:48.156335 3480 raft_consensus.cc:2393] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce5894cd69cc49c994ed89e39707de43 in current term 4: Already voted for candidate 64879c8ec3a54c119c519b91fb363693 in this term.
I20260806 07:50:48.159233 2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce5894cd69cc49c994ed89e39707de43" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64879c8ec3a54c119c519b91fb363693" is_pre_election: true
I20260806 07:50:48.160056 2859 leader_election.cc:304] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce5894cd69cc49c994ed89e39707de43; no voters: 64879c8ec3a54c119c519b91fb363693, e331606af72448db8fa1556690bdcc1f
I20260806 07:50:48.160801 3676 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:50:48.164203 3676 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 07:50:48.176102 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3528
I20260806 07:50:48.199259 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42663
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--webserver_port=34293
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {}
I20260806 07:50:48.202929 3295 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:48.203174 3295 heartbeater.cc:499] Master 127.20.202.254:34501 was elected leader, sending a full tablet report...
I20260806 07:50:48.204291 3470 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "69d5bd227a224ca69de5c4fc4d14f801" instance_seqno: 1786002646132733) as {username='slave'} at 127.20.202.194:49959; Asking this server to re-register.
I20260806 07:50:48.204988 3295 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:48.206358 3470 ts_manager.cc:194] Registered new tserver with Master: 69d5bd227a224ca69de5c4fc4d14f801 (127.20.202.194:41745)
I20260806 07:50:48.365530 3480 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [term 4 FOLLOWER]: Refusing update from remote peer 64879c8ec3a54c119c519b91fb363693: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:50:48.366545 3670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:50:48.369977 3681 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64879c8ec3a54c119c519b91fb363693. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:48.370378 3681 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:48.371943 3660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:48.372228 3660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:48.372509 2882 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [term 4 FOLLOWER]: Refusing update from remote peer 64879c8ec3a54c119c519b91fb363693: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:50:48.373167 3681 sys_catalog.cc:455] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:48.373169 3670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:50:48.373374 3681 sys_catalog.cc:458] T 00000000000000000000000000000000 P e331606af72448db8fa1556690bdcc1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:48.380661 3687 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:48.380942 3687 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:48.380802 3676 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64879c8ec3a54c119c519b91fb363693. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:48.381248 3676 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5894cd69cc49c994ed89e39707de43 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:50:48.604635 3680 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:48.605106 3680 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:48.605285 3680 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:50:48.605386 3680 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:48.622936 3680 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:48.623271 3680 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:48.644197 3680 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42663
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=34293
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:48.646780 3680 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:48.649631 3680 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:48.662763 3692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:48.664340 3694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:48.672792 3696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:48.673163 3680 server_base.cc:1061] running on GCE node
I20260806 07:50:48.673949 3680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:48.675140 3680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:48.678503 3680 hybrid_clock.cc:648] HybridClock initialized: now 1786002648678280 us; error 170 us; skew 500 ppm
I20260806 07:50:48.678841 3680 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:48.681187 3680 webserver.cc:533] Webserver started at http://127.20.202.193:34293/ using document root <none> and password file <none>
I20260806 07:50:48.682077 3680 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:48.682233 3680 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:48.688365 3680 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:50:48.691481 3702 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:48.693040 3680 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:50:48.693230 3680 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:48.694208 3680 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:48.722018 3680 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:48.722990 3680 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:48.723284 3680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:48.724335 3680 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:48.726234 3680 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:48.726368 3680 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:48.726491 3680 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:48.726580 3680 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:50:48.727043 3680 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:50:48.727156 3680 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:50:48.727238 3680 fault_injection.cc:59] injecting fault for kudu (pid 3680): FLAGS_fault_before_start (process will exit)
I20260806 07:50:48.757164 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42663
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--webserver_port=34293
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:50:48.804271 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "64879c8ec3a54c119c519b91fb363693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e331606af72448db8fa1556690bdcc1f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34501 } } peers { permanent_uuid: "ce5894cd69cc49c994ed89e39707de43" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39497 } } peers { permanent_uuid: "64879c8ec3a54c119c519b91fb363693" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35863 } } }
I20260806 07:50:48.804544 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 [sys.catalog]: This master's current role is: LEADER
W20260806 07:50:49.114809 3710 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:49.115278 3710 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:49.115434 3710 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:49.124243 3710 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:49.124495 3710 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:49.135053 3710 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40131,127.20.202.213:41949,127.20.202.214:36363,127.20.202.215:43895,127.20.202.216:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42663
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=34293
--tserver_master_addrs=127.20.202.254:34501,127.20.202.253:39497,127.20.202.252:35863
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:49.136818 3710 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:49.138808 3710 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:49.150254 3718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:49.150326 3717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:49.153445 3710 server_base.cc:1061] running on GCE node
W20260806 07:50:49.153685 3720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:49.154320 3710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:50:49.155439 3710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:50:49.156692 3710 hybrid_clock.cc:648] HybridClock initialized: now 1786002649156506 us; error 158 us; skew 500 ppm
I20260806 07:50:49.156965 3710 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:50:49.159399 3710 webserver.cc:533] Webserver started at http://127.20.202.193:34293/ using document root <none> and password file <none>
I20260806 07:50:49.160218 3710 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:49.160362 3710 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:49.166193 3710 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:50:49.169063 3726 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:49.170902 3710 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:49.171103 3710 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "06b03d73d13d4a57bac9265042308a74"
format_stamp: "Formatted at 2026-08-06 07:50:29 on dist-test-slave-g350"
I20260806 07:50:49.171845 3710 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:49.185222 3296 heartbeater.cc:499] Master 127.20.202.252:35863 was elected leader, sending a full tablet report...
I20260806 07:50:49.191294 3710 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:49.192212 3710 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:49.192502 3710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:49.193483 3710 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:49.195185 3710 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:49.195293 3710 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:49.195446 3710 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:49.195519 3710 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:49.232457 3710 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42663
I20260806 07:50:49.232549 3838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42663 every 8 connection(s)
I20260806 07:50:49.234261 3710 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:49.243372 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3710
I20260806 07:50:49.243798 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3710
I20260806 07:50:49.260823 3840 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35863
I20260806 07:50:49.261301 3840 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:49.262450 3841 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39497
I20260806 07:50:49.262611 3841 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:49.262620 3840 heartbeater.cc:507] Master 127.20.202.252:35863 requested a full tablet report, sending...
I20260806 07:50:49.263105 3839 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34501
I20260806 07:50:49.263159 3841 heartbeater.cc:507] Master 127.20.202.253:39497 requested a full tablet report, sending...
I20260806 07:50:49.263228 3839 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:49.263733 3839 heartbeater.cc:507] Master 127.20.202.254:34501 requested a full tablet report, sending...
I20260806 07:50:49.264740 2871 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:49.264848 2957 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:49.265285 3470 ts_manager.cc:194] Re-registered known tserver with Master: 06b03d73d13d4a57bac9265042308a74 (127.20.202.193:42663)
I20260806 07:50:49.266552 2957 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:50743
I20260806 07:50:49.268703 3834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:49.358803 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3710
I20260806 07:50:49.377333 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3166
I20260806 07:50:49.385926 3290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:49.483071 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3166
I20260806 07:50:49.501156 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3302
I20260806 07:50:49.509938 3425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:49.597132 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3302
I20260806 07:50:49.615428 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3438
I20260806 07:50:49.622287 3500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:49.729957 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3438
W20260806 07:50:49.752516 2942 connection.cc:573] client connection to 127.20.202.254:34501 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:50:49.752792 2942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:50:49.752986 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2825
I20260806 07:50:49.754446 2902 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:50:49.755117 2942 consensus_peers.cc:597] T 00000000000000000000000000000000 P 64879c8ec3a54c119c519b91fb363693 -> Peer e331606af72448db8fa1556690bdcc1f (127.20.202.254:34501): Couldn't send request to peer e331606af72448db8fa1556690bdcc1f. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:34501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:50:49.753500 2909 master_runner.cc:187] Error getting master registration for 127.20.202.254:34501: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260806 07:50:49.760274 2909 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:49.866485 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2825
I20260806 07:50:49.889204 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2909
I20260806 07:50:49.890555 2987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:50.003347 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2909
Shutting down hive metastore.
2026-08-06T07:50:50Z chronyd exiting
2026-08-06T07:50:50Z chronyd exiting
2026-08-06T07:50:50Z chronyd exiting
2026-08-06T07:50:50Z chronyd exiting
2026-08-06T07:50:50Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/5 (39750 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/6
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:50:50 dist-test-slave-g350 krb5kdc[3868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:50:50 dist-test-slave-g350 krb5kdc[3868](info): set up 2 sockets
Aug 06 07:50:50 dist-test-slave-g350 krb5kdc[3868](info): commencing operation
krb5kdc: starting...
W20260806 07:50:52.525445 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.051s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:50:52 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:50:52.782596 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:50:53.121901 3882 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:53.122293 3882 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:53.122452 3882 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:53.130684 3882 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:53.130841 3882 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:53.130976 3882 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:53.131063 3882 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:53.141821 3882 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:53.143507 3882 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:53.144912 3882 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:53.153472 3888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:53.153537 3889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:53.153695 3891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:53.154440 3882 server_base.cc:1061] running on GCE node
I20260806 07:50:53.155130 3882 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:53.155273 3882 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:53.155331 3882 hybrid_clock.cc:648] HybridClock initialized: now 1786002653155313 us; error 0 us; skew 500 ppm
I20260806 07:50:53.155646 3882 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:53 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:53.159140 3882 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:50:53.161195 3882 webserver.cc:533] Webserver started at http://127.20.202.254:44435/ using document root <none> and password file <none>
I20260806 07:50:53.162017 3882 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:53.162122 3882 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:53.162439 3882 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:53.164984 3882 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "34335818920843458fa8203dca58c94f"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:53.165870 3882 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "34335818920843458fa8203dca58c94f"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:53.171635 3882 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:50:53.175266 3897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:53.176709 3882 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:50:53.176906 3882 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "34335818920843458fa8203dca58c94f"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:53.177083 3882 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:53.215524 3882 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:53.221477 3882 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:53.221844 3882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:53.246438 3882 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:46631
I20260806 07:50:53.246457 3948 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:46631 every 8 connection(s)
I20260806 07:50:53.248026 3882 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:53.252292 3949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:50:53.256220 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3882
I20260806 07:50:53.256428 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:50:53.258157 3949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:50:53 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
I20260806 07:50:53.282790 3949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
W20260806 07:50:53.284027 3901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111)
W20260806 07:50:53.285782 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40617: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
I20260806 07:50:53.335122 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } attempt: 1
W20260806 07:50:53.337311 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40617: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:50:53.361174 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40617
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:50:53.405675 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } attempt: 2
W20260806 07:50:53.408063 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40617: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111)
I20260806 07:50:53.499308 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } attempt: 3
W20260806 07:50:53.501549 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40617: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111)
I20260806 07:50:53.644832 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } attempt: 4
W20260806 07:50:53.647152 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40617: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111)
W20260806 07:50:53.702240 3958 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:53.702634 3958 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:53.702790 3958 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:53.711134 3958 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:53.711277 3958 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:53.711404 3958 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:53.711490 3958 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:53.722394 3958 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:53.724153 3958 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:53.725663 3958 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:53.734174 3965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:53.734227 3964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:53.734793 3967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:53.735212 3958 server_base.cc:1061] running on GCE node
I20260806 07:50:53.736007 3958 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:53.736207 3958 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:53.736312 3958 hybrid_clock.cc:648] HybridClock initialized: now 1786002653736272 us; error 0 us; skew 500 ppm
I20260806 07:50:53.736653 3958 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:53 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:53.740048 3958 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:50:53.742317 3958 webserver.cc:533] Webserver started at http://127.20.202.253:41407/ using document root <none> and password file <none>
I20260806 07:50:53.743165 3958 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:53.743310 3958 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:53.743685 3958 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:53.746263 3958 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "ce9b20436581491dab395958d70e4796"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:53.747067 3958 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "ce9b20436581491dab395958d70e4796"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:53.752528 3958 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:50:53.756214 3973 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:53.757757 3958 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:53.757982 3958 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "ce9b20436581491dab395958d70e4796"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:53.758193 3958 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:53.776999 3958 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:53.781184 3958 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:53.781529 3958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:53.802431 3958 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:40617
I20260806 07:50:53.802453 4024 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:40617 every 8 connection(s)
I20260806 07:50:53.804029 3958 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:50:53.804383 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3958
I20260806 07:50:53.804595 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:50:53.809794 4025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:50:53.817157 4025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:50:53 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:50:53.840765 4025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:50:53 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:50:53.859097 4025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:50:53.860440 3976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
W20260806 07:50:53.862210 4025 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:50:53.911636 4025 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
W20260806 07:50:53.913991 4025 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:53.915151 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:46631
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:50:53.946470 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } attempt: 5
Aug 06 07:50:53 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:50:53.961846 3949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:50:53.963969 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:53.982227 4025 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 2
W20260806 07:50:53.984454 4025 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:54.015242 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 1
W20260806 07:50:54.017459 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:54.075747 4025 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 3
W20260806 07:50:54.078269 4025 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:54.085697 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 2
W20260806 07:50:54.087741 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:54.194020 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 3
W20260806 07:50:54.196086 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:54.221549 4025 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 4
W20260806 07:50:54.223817 4025 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
W20260806 07:50:54.269942 4035 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:54.270282 4035 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:54.270431 4035 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:54.278925 4035 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:54.279023 4035 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:54.279155 4035 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:54.279227 4035 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:54.290163 4035 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:54.291859 4035 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:54.293236 4035 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:54.301625 4042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:54.301697 4041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:54.302714 4044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:54.303557 4035 server_base.cc:1061] running on GCE node
I20260806 07:50:54.304349 4035 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:54.304529 4035 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:54.304641 4035 hybrid_clock.cc:648] HybridClock initialized: now 1786002654304601 us; error 0 us; skew 500 ppm
I20260806 07:50:54.305006 4035 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002654, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:54.308622 4035 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:50:54.310663 4035 webserver.cc:533] Webserver started at http://127.20.202.252:33569/ using document root <none> and password file <none>
I20260806 07:50:54.311409 4035 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:54.311553 4035 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:54.311902 4035 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:54.314368 4035 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "f25f80bac722409a92ecca195df846d9"
format_stamp: "Formatted at 2026-08-06 07:50:54 on dist-test-slave-g350"
I20260806 07:50:54.315168 4035 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "f25f80bac722409a92ecca195df846d9"
format_stamp: "Formatted at 2026-08-06 07:50:54 on dist-test-slave-g350"
I20260806 07:50:54.320680 4035 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 07:50:54.324443 4050 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:54.326179 4035 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:50:54.326400 4035 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "f25f80bac722409a92ecca195df846d9"
format_stamp: "Formatted at 2026-08-06 07:50:54 on dist-test-slave-g350"
I20260806 07:50:54.326650 4035 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:54.349076 4035 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:54.356873 4035 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:54.357210 4035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:54.373327 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 4
W20260806 07:50:54.375775 3949 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111)
I20260806 07:50:54.379132 4035 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:46631
I20260806 07:50:54.379218 4101 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:46631 every 8 connection(s)
I20260806 07:50:54.380720 4035 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:50:54.385265 4102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:50:54.389782 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4035
I20260806 07:50:54.389991 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:50:54.392199 4102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002654, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:50:54.415385 4102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002654, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:50:54.430070 4102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002654, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:50:54.450003 4102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Bootstrap starting.
I20260806 07:50:54.453703 4102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Neither blocks nor log segments found. Creating new log.
I20260806 07:50:54.454800 4102 log.cc:826] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:54.457372 4102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: No bootstrap required, opened a new log
I20260806 07:50:54.463181 4102 raft_consensus.cc:359] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.463539 4102 raft_consensus.cc:385] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:50:54.463673 4102 raft_consensus.cc:740] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25f80bac722409a92ecca195df846d9, State: Initialized, Role: FOLLOWER
I20260806 07:50:54.464354 4102 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.465263 4112 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:54.465499 4112 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:54.466578 4102 sys_catalog.cc:565] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:50:54.478627 4123 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:50:54.478786 4123 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:50:54.473632 4035 master_runner.cc:187] Error getting master registration for 127.20.202.254:46631: OK, instance_id { permanent_uuid: "34335818920843458fa8203dca58c94f" instance_seqno: 1786002653240749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:50:54.482795 4035 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:54.523146 4025 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 5
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:50:54.542729 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Bootstrap starting.
I20260806 07:50:54.546406 4025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Neither blocks nor log segments found. Creating new log.
I20260806 07:50:54.547601 4025 log.cc:826] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:54.551308 4025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: No bootstrap required, opened a new log
I20260806 07:50:54.558971 4025 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.559316 4025 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:50:54.559453 4025 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9b20436581491dab395958d70e4796, State: Initialized, Role: FOLLOWER
I20260806 07:50:54.560158 4025 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.561087 4126 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:54.561354 4126 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:54.562515 4025 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:50:54.574316 4137 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:50:54.574532 4137 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:50:54.571282 3958 master_runner.cc:187] Error getting master registration for 127.20.202.254:46631: OK, instance_id { permanent_uuid: "34335818920843458fa8203dca58c94f" instance_seqno: 1786002653240749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:50:54.577000 3958 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:54.609848 4056 tablet.cc:2406] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:50:54.653059 3949 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } attempt: 5
Aug 06 07:50:54 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002653, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:50:54.678534 3949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap starting.
I20260806 07:50:54.682899 3949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Neither blocks nor log segments found. Creating new log.
I20260806 07:50:54.684401 3949 log.cc:826] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:54.687711 3949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: No bootstrap required, opened a new log
I20260806 07:50:54.696981 3949 raft_consensus.cc:359] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.697335 3949 raft_consensus.cc:385] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:50:54.697471 3949 raft_consensus.cc:740] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34335818920843458fa8203dca58c94f, State: Initialized, Role: FOLLOWER
I20260806 07:50:54.698284 3949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.699350 4140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:54.699728 4140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:54.700678 3949 sys_catalog.cc:565] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:54.720754 3882 master_runner.cc:264] No leader master found from master 34335818920843458fa8203dca58c94f
I20260806 07:50:54.720922 3882 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:50:54.721719 4151 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:50:54.721894 4151 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:50:54.725629 3903 tablet.cc:2406] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:50:54.784461 3979 tablet.cc:2406] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:50:54.926796 4126 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:50:54.927095 4126 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.928777 4126 leader_election.cc:290] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34335818920843458fa8203dca58c94f (127.20.202.254:46631), f25f80bac722409a92ecca195df846d9 (127.20.202.252:46631)
I20260806 07:50:54.930120 4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25f80bac722409a92ecca195df846d9" is_pre_election: true
I20260806 07:50:54.930634 4077 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9b20436581491dab395958d70e4796 in term 0.
I20260806 07:50:54.931979 3924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34335818920843458fa8203dca58c94f" is_pre_election: true
I20260806 07:50:54.932432 3924 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9b20436581491dab395958d70e4796 in term 0.
I20260806 07:50:54.933336 3976 leader_election.cc:304] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce9b20436581491dab395958d70e4796, f25f80bac722409a92ecca195df846d9; no voters:
I20260806 07:50:54.933905 4126 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:50:54.934062 4126 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:50:54.934162 4126 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:50:54.938179 4126 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.939390 3924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34335818920843458fa8203dca58c94f"
I20260806 07:50:54.939596 3924 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:50:54.943182 3924 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9b20436581491dab395958d70e4796 in term 1.
I20260806 07:50:54.943809 3977 leader_election.cc:304] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34335818920843458fa8203dca58c94f, ce9b20436581491dab395958d70e4796; no voters:
I20260806 07:50:54.945506 4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25f80bac722409a92ecca195df846d9"
I20260806 07:50:54.945789 4077 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:50:54.949151 4077 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9b20436581491dab395958d70e4796 in term 1.
I20260806 07:50:54.950044 4126 leader_election.cc:290] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 1 election: Requested vote from peers 34335818920843458fa8203dca58c94f (127.20.202.254:46631), f25f80bac722409a92ecca195df846d9 (127.20.202.252:46631)
I20260806 07:50:54.950474 4126 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:50:54.950917 4126 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 LEADER]: Becoming Leader. State: Replica: ce9b20436581491dab395958d70e4796, State: Running, Role: LEADER
I20260806 07:50:54.951668 4126 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:54.969010 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:54.969494 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:54.976658 4155 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:50:54.979143 4155 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:50:54.994554 4077 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 1 FOLLOWER]: Refusing update from remote peer ce9b20436581491dab395958d70e4796: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:50:54.994565 3924 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Refusing update from remote peer ce9b20436581491dab395958d70e4796: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:50:54.995678 4153 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:50:54.996446 4153 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:50:55.023335 4140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:55.023756 4140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:55.028729 4156 mvcc.cc:204] Tried to move back new op lower bound from 7315466874846064640 to 7315466874705604608. Current Snapshot: MvccSnapshot[applied={T|T < 7315466874846064640}]
I20260806 07:50:55.056881 4126 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:55.057307 4126 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:55.060050 4155 catalog_manager.cc:1383] Generated new cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:50:55.060179 4155 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:50:55.060819 4140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:55.061053 4140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:55.066581 4157 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:55.066839 4157 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:55.068689 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:55.068902 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:55.079033 4157 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:55.079269 4157 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:55.081888 4155 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:50:55.083709 4155 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:50:55.110819 4155 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Generated new TSK 0
I20260806 07:50:55.111999 4155 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
W20260806 07:50:55.253377 3945 messenger.cc:194] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.202.254:46631
I20260806 07:50:55.253783 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:50:55.386569 4098 messenger.cc:194] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.202.252:46631
I20260806 07:50:55.482231 4123 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:50:55.482368 4123 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: loading cluster ID for follower catalog manager: success
I20260806 07:50:55.486217 4123 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: acquiring CA information for follower catalog manager: success
I20260806 07:50:55.488629 4123 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:50:55.607484 4170 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:55.607816 4170 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:55.607999 4170 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:55.616729 4170 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:55.616955 4170 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:55.627843 4170 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:55.629683 4170 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:55.631134 4170 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:55.641615 4177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:55.641654 4176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:55.641908 4179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:55.643090 4170 server_base.cc:1061] running on GCE node
I20260806 07:50:55.643739 4170 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:55.643939 4170 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:55.644033 4170 hybrid_clock.cc:648] HybridClock initialized: now 1786002655644013 us; error 0 us; skew 500 ppm
I20260806 07:50:55.644373 4170 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:55 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:55.647943 4170 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:50:55.649531 4170 webserver.cc:533] Webserver started at http://127.20.202.193:40933/ using document root <none> and password file <none>
I20260806 07:50:55.650282 4170 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:55.650417 4170 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:55.650795 4170 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:55.653172 4170 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:50:55.653976 4170 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:50:55.660032 4170 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:50:55.663805 4185 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:55.665377 4170 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:50:55.665567 4170 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:50:55.665845 4170 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:55.681684 4170 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:55.685774 4170 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:55.686107 4170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:55.687160 4170 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:55.688851 4170 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:55.688967 4170 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:55.689152 4170 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:55.689241 4170 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:55.724399 4151 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:50:55.724529 4151 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: loading cluster ID for follower catalog manager: success
I20260806 07:50:55.727824 4151 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: acquiring CA information for follower catalog manager: success
I20260806 07:50:55.728428 4170 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37251
I20260806 07:50:55.728494 4297 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37251 every 8 connection(s)
I20260806 07:50:55.730020 4170 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:55.730118 4151 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:50:55.731397 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4170
I20260806 07:50:55.731652 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:50:55 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
Aug 06 07:50:55 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.194@KRBTEST.COM" created.
Aug 06 07:50:55 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002655, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:50:55.785878 4298 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:50:55.786293 4298 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:55.786886 4300 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:50:55.786974 4299 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:50:55.787071 4300 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:55.787163 4299 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:55.787858 4298 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:50:55.788094 4300 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:50:55.788388 4299 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:50:55.790426 3990 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:37251)
I20260806 07:50:55.790729 4066 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:37251)
I20260806 07:50:55.791743 3914 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:37251)
I20260806 07:50:55.792600 3990 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:46527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:50:55.861370 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:50:56.216125 4312 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:56.216418 4312 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:56.216612 4312 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:56.225103 4312 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:56.225325 4312 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:50:56.236076 4312 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:56.237767 4312 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:56.239171 4312 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:56.249177 4319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:56.249974 4321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:56.250852 4317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:56.251475 4312 server_base.cc:1061] running on GCE node
I20260806 07:50:56.252115 4312 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:56.252306 4312 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:56.252410 4312 hybrid_clock.cc:648] HybridClock initialized: now 1786002656252384 us; error 0 us; skew 500 ppm
I20260806 07:50:56.252734 4312 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:56.256819 4312 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:50:56.258803 4312 webserver.cc:533] Webserver started at http://127.20.202.194:33443/ using document root <none> and password file <none>
I20260806 07:50:56.259721 4312 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:56.259853 4312 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:56.260231 4312 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:56.262801 4312 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "dba3c45e12cd4dc8b72b29b186a5137d"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:50:56.263746 4312 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "dba3c45e12cd4dc8b72b29b186a5137d"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:50:56.269464 4312 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:50:56.272944 4327 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:56.274390 4312 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:56.274590 4312 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "dba3c45e12cd4dc8b72b29b186a5137d"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:50:56.274772 4312 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:56.295835 4312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:56.300041 4312 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:56.300354 4312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:56.301345 4312 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:56.303088 4312 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:56.303162 4312 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:56.303313 4312 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:56.303386 4312 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:56.339174 4312 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:34111
I20260806 07:50:56.339363 4439 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:34111 every 8 connection(s)
I20260806 07:50:56.340667 4312 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:50:56.346243 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4312
I20260806 07:50:56.346493 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:50:56.383527 4440 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:50:56.383980 4440 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:56.385159 4440 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:50:56.387809 4066 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:34111)
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.195@KRBTEST.COM" created.
I20260806 07:50:56.397703 4441 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:50:56.397869 4441 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:56.398423 4441 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:50:56.399782 3914 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:34111)
I20260806 07:50:56.399963 4442 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:50:56.400098 4442 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:56.400557 4442 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:50:56.401575 3990 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:34111)
I20260806 07:50:56.402500 3990 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:57383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:50:56.474561 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:50:56.794965 4300 heartbeater.cc:499] Master 127.20.202.253:40617 was elected leader, sending a full tablet report...
W20260806 07:50:56.819141 4454 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:56.819535 4454 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:56.819705 4454 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:56.828312 4454 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:56.828573 4454 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:50:56.840399 4454 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:56.842221 4454 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:56.843703 4454 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:56.854911 4461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:56.854945 4463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:56.854969 4459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:56.855877 4454 server_base.cc:1061] running on GCE node
I20260806 07:50:56.856499 4454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:56.856693 4454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:56.856793 4454 hybrid_clock.cc:648] HybridClock initialized: now 1786002656856769 us; error 0 us; skew 500 ppm
I20260806 07:50:56.857117 4454 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:56.860896 4454 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:50:56.862710 4454 webserver.cc:533] Webserver started at http://127.20.202.195:35427/ using document root <none> and password file <none>
I20260806 07:50:56.863545 4454 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:56.863699 4454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:56.864135 4454 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:50:56.866816 4454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "c4dabe24cfd342eb8cf974dd42356343"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:50:56.867712 4454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "c4dabe24cfd342eb8cf974dd42356343"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:50:56.873687 4454 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:50:56.877075 4469 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:56.878626 4454 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:50:56.878813 4454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "c4dabe24cfd342eb8cf974dd42356343"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:50:56.879007 4454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:56.906898 4454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:56.911964 4454 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:56.912325 4454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:56.913463 4454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:56.915372 4454 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:56.915493 4454 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:56.915685 4454 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:56.915771 4454 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:56.954809 4454 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:41319
I20260806 07:50:56.954844 4581 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:41319 every 8 connection(s)
I20260806 07:50:56.956465 4454 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:50:56.959983 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4454
I20260806 07:50:56.960232 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:50:56.989140 4582 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:50:56.989634 4582 heartbeater.cc:461] Registering TS with master...
Aug 06 07:50:56 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002656, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:50:56.990852 4582 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:50:56.992940 4066 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:41319)
I20260806 07:50:56.995566 4584 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:50:56.995756 4584 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:56.996416 4584 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:50:56.997781 3990 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:41319)
I20260806 07:50:56.998792 3990 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:50005
I20260806 07:50:57.000005 4583 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:50:57.000185 4583 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:57.001062 4583 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:50:57.002215 3914 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:41319)
I20260806 07:50:57.013033 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:57.021138 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:50:57.021317 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3882
I20260806 07:50:57.022218 3944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:57.132473 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3882
I20260806 07:50:57.156694 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--webserver_interface=127.20.202.254
--webserver_port=44435
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:50:57.402302 4441 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:46631 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:46631: connect: Connection refused (error 111)
I20260806 07:50:57.405076 4442 heartbeater.cc:499] Master 127.20.202.253:40617 was elected leader, sending a full tablet report...
W20260806 07:50:57.513827 3977 consensus_peers.cc:597] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 -> Peer 34335818920843458fa8203dca58c94f (127.20.202.254:46631): Couldn't send request to peer 34335818920843458fa8203dca58c94f. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:46631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:50:57.518069 4593 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:57.518430 4593 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:57.518579 4593 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:57.527458 4593 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:50:57.527609 4593 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:57.527735 4593 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:50:57.527820 4593 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:50:57.539348 4593 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=44435
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:57.541298 4593 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:57.542759 4593 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:57.551440 4602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:57.551708 4604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:57.551749 4601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:57.552762 4593 server_base.cc:1061] running on GCE node
I20260806 07:50:57.553535 4593 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:57.553771 4593 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:57.553874 4593 hybrid_clock.cc:648] HybridClock initialized: now 1786002657553832 us; error 0 us; skew 500 ppm
I20260806 07:50:57.554245 4593 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:57 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002657, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:57.557899 4593 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:50:57.560014 4593 webserver.cc:533] Webserver started at http://127.20.202.254:44435/ using document root <none> and password file <none>
I20260806 07:50:57.560786 4593 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:57.560945 4593 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:57.566704 4593 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:50:57.570016 4610 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:57.571681 4593 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 07:50:57.571878 4593 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "34335818920843458fa8203dca58c94f"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:50:57.572685 4593 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:57.598129 4593 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:57.602479 4593 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:57.602818 4593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:57.625504 4593 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:46631
I20260806 07:50:57.625523 4663 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:46631 every 8 connection(s)
I20260806 07:50:57.627336 4593 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:50:57.632087 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4593
I20260806 07:50:57.632419 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4170
I20260806 07:50:57.635218 4664 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:50:57.639264 4664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap starting.
Aug 06 07:50:57 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:50:57.652590 4293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:57.659880 4664 log.cc:826] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:50:57.668958 4664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:50:57.669661 4664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap complete.
I20260806 07:50:57.676498 4664 raft_consensus.cc:359] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:57.677215 4664 raft_consensus.cc:740] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34335818920843458fa8203dca58c94f, State: Initialized, Role: FOLLOWER
I20260806 07:50:57.678061 4664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:50:57.678977 4670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:57.679282 4670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:57.680091 4664 sys_catalog.cc:565] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:50:57.690459 4681 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:50:57.690645 4681 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: loading cluster ID for follower catalog manager: success
I20260806 07:50:57.694713 4681 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: acquiring CA information for follower catalog manager: success
I20260806 07:50:57.697217 4681 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:50:57.723935 4101 acceptor_pool.cc:311] unable to collect diagnostics on RPC socket 127.20.202.252:46631: Invalid argument: socket address is ambiguous
I20260806 07:50:57.754729 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4170
I20260806 07:50:57.778861 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37251
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--webserver_port=40933
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:50:57.934347 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:57.934666 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:57.951650 4670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:57.951982 4670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:50:58.002099 4584 heartbeater.cc:499] Master 127.20.202.253:40617 was elected leader, sending a full tablet report...
I20260806 07:50:58.014103 4628 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c4dabe24cfd342eb8cf974dd42356343" instance_seqno: 1786002656945797) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:52639; Asking this server to re-register.
I20260806 07:50:58.015009 4583 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:58.016916 4628 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:41319)
W20260806 07:50:58.147485 4687 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:58.147871 4687 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:58.148042 4687 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:58.157331 4687 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:58.157624 4687 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:58.169209 4687 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40933
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:58.171130 4687 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:58.172593 4687 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:58.182838 4696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:58.182868 4697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:58.183578 4699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:58.183840 4687 server_base.cc:1061] running on GCE node
I20260806 07:50:58.184471 4687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:58.184703 4687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:58.184815 4687 hybrid_clock.cc:648] HybridClock initialized: now 1786002658184793 us; error 0 us; skew 500 ppm
I20260806 07:50:58.185145 4687 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:58 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002658, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:58.188582 4687 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:50:58.190332 4687 webserver.cc:533] Webserver started at http://127.20.202.193:40933/ using document root <none> and password file <none>
I20260806 07:50:58.191123 4687 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:58.191282 4687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:58.196730 4687 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 07:50:58.199517 4705 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:58.200815 4687 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:50:58.200992 4687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:50:58.201808 4687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:58.223891 4687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:58.227836 4687 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:58.228147 4687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:58.229252 4687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:58.231137 4687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:58.231266 4687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:58.231400 4687 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:58.231498 4687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:58.268226 4687 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37251
I20260806 07:50:58.268285 4819 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37251 every 8 connection(s)
I20260806 07:50:58.269873 4687 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:58.277838 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4687
Aug 06 07:50:58 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002658, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:50:58 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002658, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:50:58 dist-test-slave-g350 krb5kdc[3868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002658, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:50:58.300725 4821 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:50:58.301213 4820 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:50:58.301354 4820 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:58.301548 4822 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:50:58.301702 4822 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:58.301213 4821 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:58.302816 4821 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:50:58.302881 4820 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:50:58.302913 4822 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:50:58.304874 4628 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:37251)
I20260806 07:50:58.305239 4066 ts_manager.cc:194] Re-registered known tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:37251)
I20260806 07:50:58.306185 3990 ts_manager.cc:194] Re-registered known tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:37251)
I20260806 07:50:58.307153 3990 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:50851
I20260806 07:50:58.334514 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4687
W20260806 07:50:58.334933 4832 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:50:58.331182 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.20.202.193:37251)
0806 07:50:58.331879 (+ 697us) server_negotiation.cc:207] Beginning negotiation
0806 07:50:58.331888 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:50:58.331929 (+ 41us) server_negotiation.cc:408] Connection header received
0806 07:50:58.332194 (+ 265us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:50:58.332206 (+ 12us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:50:58.332456 (+ 250us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:50:58.332626 (+ 170us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:50:58.333858 (+ 1232us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60988: BlockingRecv error: recv got EOF from 127.0.0.1:60988 (error 108)
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":328,"threads_started":1}
W20260806 07:50:58.336653 4832 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:50:58.335378 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.20.202.193:37251)
0806 07:50:58.335666 (+ 288us) server_negotiation.cc:207] Beginning negotiation
0806 07:50:58.335670 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:50:58.335685 (+ 15us) server_negotiation.cc:408] Connection header received
0806 07:50:58.335726 (+ 41us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:50:58.335729 (+ 3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:50:58.335780 (+ 51us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:50:58.335861 (+ 81us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:50:58.336454 (+ 593us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32772: BlockingRecv error: recv got EOF from 127.0.0.1:32772 (error 108)
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":175}
W20260806 07:50:58.336723 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4687: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:37251: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:50:58.336966 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4687
I20260806 07:50:58.356334 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37251
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--webserver_port=40933
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:50:58.371584 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:50:58.371866 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:50:58.410023 4441 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:50:58.411338 4628 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dba3c45e12cd4dc8b72b29b186a5137d" instance_seqno: 1786002656329926) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:33319; Asking this server to re-register.
I20260806 07:50:58.411978 4441 heartbeater.cc:461] Registering TS with master...
I20260806 07:50:58.413266 4628 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:34111)
W20260806 07:50:58.633807 4660 messenger.cc:194] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.202.254:46631
W20260806 07:50:58.714527 4833 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:58.714861 4833 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:58.715052 4833 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:50:58.715121 4833 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:58.726002 4833 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:58.726225 4833 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:58.737128 4833 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40933
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:58.738897 4833 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:58.740403 4833 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:58.753909 4839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:58.753877 4840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:58.754252 4842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:58.755430 4833 server_base.cc:1061] running on GCE node
I20260806 07:50:58.756045 4833 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:58.756245 4833 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:58.756368 4833 hybrid_clock.cc:648] HybridClock initialized: now 1786002658756312 us; error 0 us; skew 500 ppm
I20260806 07:50:58.756660 4833 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:58 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002658, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:58.760149 4833 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:50:58.761853 4833 webserver.cc:533] Webserver started at http://127.20.202.193:40933/ using document root <none> and password file <none>
I20260806 07:50:58.762704 4833 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:58.762815 4833 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:58.768635 4833 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 07:50:58.771639 4848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:58.773038 4833 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:50:58.773240 4833 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:50:58.774011 4833 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:58.807453 4833 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:58.812615 4833 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:58.812947 4833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:58.814019 4833 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:58.815611 4833 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:58.815702 4833 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:58.815784 4833 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:58.815820 4833 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:50:58.816279 4833 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:50:58.816346 4833 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:50:58.816437 4833 fault_injection.cc:59] injecting fault for kudu (pid 4833): FLAGS_fault_before_start (process will exit)
I20260806 07:50:58.833169 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37251
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--webserver_port=40933
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:50:59.189028 4856 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:50:59.189350 4856 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:50:59.189447 4856 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:50:59.197618 4856 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:50:59.197798 4856 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:50:59.208178 4856 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40933
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:50:59.209817 4856 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:50:59.211151 4856 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:50:59.220914 4861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:59.221045 4862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:50:59.221228 4864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:50:59.222226 4856 server_base.cc:1061] running on GCE node
I20260806 07:50:59.222771 4856 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:50:59.222908 4856 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:50:59.222971 4856 hybrid_clock.cc:648] HybridClock initialized: now 1786002659222954 us; error 0 us; skew 500 ppm
I20260806 07:50:59.223222 4856 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:50:59 dist-test-slave-g350 krb5kdc[3868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002659, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:50:59.226583 4856 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:50:59.228227 4856 webserver.cc:533] Webserver started at http://127.20.202.193:40933/ using document root <none> and password file <none>
I20260806 07:50:59.228998 4856 fs_manager.cc:362] Metadata directory not provided
I20260806 07:50:59.229146 4856 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:50:59.234899 4856 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:50:59.237757 4870 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:59.239254 4856 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:50:59.239462 4856 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:50:59.240279 4856 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:50:59.256304 4856 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:50:59.260393 4856 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:50:59.260697 4856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:50:59.261750 4856 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:50:59.263475 4856 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:50:59.263614 4856 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:59.263742 4856 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:50:59.263830 4856 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:50:59.299948 4856 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37251
I20260806 07:50:59.300004 4985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37251 every 8 connection(s)
I20260806 07:50:59.301481 4856 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:50:59.308307 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4856
I20260806 07:50:59.308707 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4856
W20260806 07:50:59.322901 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4856: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:37251: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:50:59.323112 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4856
I20260806 07:50:59.341526 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4312
W20260806 07:50:59.342168 4991 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:50:59.304909 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:44005 (local address 127.20.202.254:46631)
0806 07:50:59.320396 (+ 15487us) server_negotiation.cc:207] Beginning negotiation
0806 07:50:59.320406 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0806 07:50:59.323680 (+ 3274us) server_negotiation.cc:408] Connection header received
0806 07:50:59.341212 (+ 17532us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:44005: BlockingRecv error: recv got EOF from 127.20.202.193:44005 (error 108)
Metrics: {"server-negotiator.queue_time_us":15291,"thread_start_us":15501,"threads_started":1}
W20260806 07:50:59.342181 4990 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:50:59.304815 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:57787 (local address 127.20.202.252:46631)
0806 07:50:59.325220 (+ 20405us) server_negotiation.cc:207] Beginning negotiation
0806 07:50:59.325228 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:50:59.341195 (+ 15967us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:57787: BlockingRecv error: recv got EOF from 127.20.202.193:57787 (error 108)
Metrics: {"server-negotiator.queue_time_us":20275,"thread_start_us":20105,"threads_started":1}
W20260806 07:50:59.344473 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4312: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:34111: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:50:59.344630 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4312
I20260806 07:50:59.362299 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4454
W20260806 07:50:59.365185 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4454: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:41319: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:50:59.365336 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4454
I20260806 07:50:59.383217 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4593
W20260806 07:50:59.385711 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4593: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:46631: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:50:59.385907 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4593
W20260806 07:50:59.385908 4991 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:50:59.384168 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:52620 (local address 127.20.202.254:46631)
0806 07:50:59.384380 (+ 212us) server_negotiation.cc:207] Beginning negotiation
0806 07:50:59.384386 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:50:59.384481 (+ 95us) server_negotiation.cc:408] Connection header received
0806 07:50:59.384725 (+ 244us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:50:59.384731 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:50:59.384798 (+ 67us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:50:59.384911 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:50:59.385663 (+ 752us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52620: BlockingRecv error: recv got EOF from 127.0.0.1:52620 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 07:50:59.405076 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3958
I20260806 07:50:59.406569 4020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:59.484731 4035 master_runner.cc:187] Error getting master registration for 127.20.202.254:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.254:46631: connect: Connection refused (error 111),
I20260806 07:50:59.484969 4035 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:50:59.520819 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3958
I20260806 07:50:59.543918 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4035
I20260806 07:50:59.545118 4097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:50:59.656860 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4035
Aug 06 07:50:59 dist-test-slave-g350 krb5kdc[5002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:50:59 dist-test-slave-g350 krb5kdc[5002](info): set up 2 sockets
Aug 06 07:50:59 dist-test-slave-g350 krb5kdc[5002](info): commencing operation
krb5kdc: starting...
W20260806 07:51:01.747017 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.063s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:51:01 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:01.972672 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:02.315891 5016 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:02.316290 5016 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:02.316447 5016 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:02.324836 5016 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:02.325019 5016 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:02.325168 5016 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:02.325254 5016 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:02.336300 5016 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:02.338152 5016 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:02.339572 5016 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:02.348459 5022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:02.348529 5021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:02.349188 5024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:02.349995 5016 server_base.cc:1061] running on GCE node
I20260806 07:51:02.350760 5016 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:02.350953 5016 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:02.351059 5016 hybrid_clock.cc:648] HybridClock initialized: now 1786002662351018 us; error 0 us; skew 500 ppm
I20260806 07:51:02.351402 5016 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:02 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:02.355014 5016 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:02.356889 5016 webserver.cc:533] Webserver started at http://127.20.202.254:33241/ using document root <none> and password file <none>
I20260806 07:51:02.357693 5016 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:02.357790 5016 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:02.363005 5016 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 07:51:02.365856 5030 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:02.367336 5016 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:02.367511 5016 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "34335818920843458fa8203dca58c94f"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:51:02.368266 5016 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:02.383201 5016 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:02.392735 5016 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:02.393054 5016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:02.414970 5016 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:46631
I20260806 07:51:02.414978 5082 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:46631 every 8 connection(s)
I20260806 07:51:02.416708 5016 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:02.417912 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5016
I20260806 07:51:02.418154 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:51:02.427174 5084 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:02.431084 5084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:02.450203 5084 log.cc:826] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:02.459313 5084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:02.459869 5084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap complete.
I20260806 07:51:02.467425 5084 raft_consensus.cc:359] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:02.468282 5084 raft_consensus.cc:740] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34335818920843458fa8203dca58c94f, State: Initialized, Role: FOLLOWER
I20260806 07:51:02.469172 5084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:02.470055 5089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:02.470361 5089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:02.471223 5084 sys_catalog.cc:565] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:02.482218 5100 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:51:02.482405 5100 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: loading cluster ID for follower catalog manager: success
I20260806 07:51:02.486869 5100 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: acquiring CA information for follower catalog manager: success
I20260806 07:51:02.489641 5100 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:51:02 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:02.512830 5016 master_runner.cc:187] Error getting master registration for 127.20.202.253:40617: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40617: connect: Connection refused (error 111),
I20260806 07:51:02.518525 5016 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:02.535148 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40617
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:02.881088 5106 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:02.881417 5106 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:02.881583 5106 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:02.890100 5106 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:02.890199 5106 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:02.890322 5106 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:02.890394 5106 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:02.901185 5106 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:02.902875 5106 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:02.904305 5106 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:02.913043 5112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:02.913116 5111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:02.914424 5114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:02.914497 5106 server_base.cc:1061] running on GCE node
I20260806 07:51:02.915328 5106 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:02.915505 5106 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:02.915609 5106 hybrid_clock.cc:648] HybridClock initialized: now 1786002662915570 us; error 0 us; skew 500 ppm
I20260806 07:51:02.915992 5106 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:02 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:02.919767 5106 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:51:02.921741 5106 webserver.cc:533] Webserver started at http://127.20.202.253:33093/ using document root <none> and password file <none>
I20260806 07:51:02.922458 5106 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:02.922611 5106 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:02.927899 5106 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 07:51:02.930699 5120 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:02.932025 5106 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:02.932211 5106 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "ce9b20436581491dab395958d70e4796"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:51:02.932911 5106 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:02.948271 5106 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:02.952154 5106 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:02.952414 5106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:02.973307 5106 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:40617
I20260806 07:51:02.973330 5171 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:40617 every 8 connection(s)
I20260806 07:51:02.975023 5106 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:51:02.980057 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5106
I20260806 07:51:02.980271 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:51:02.983080 5172 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:02.988085 5172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:03.006693 5172 log.cc:826] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:03.016312 5172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:03.016844 5172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Bootstrap complete.
I20260806 07:51:03.023756 5172 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.024425 5172 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9b20436581491dab395958d70e4796, State: Initialized, Role: FOLLOWER
I20260806 07:51:03.025259 5172 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.026201 5177 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:03.026616 5177 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:03.027410 5172 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:03.038079 5188 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:51:03.038240 5188 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: loading cluster ID for follower catalog manager: success
I20260806 07:51:03.042557 5188 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: acquiring CA information for follower catalog manager: success
I20260806 07:51:03.045223 5188 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:03.068840 5106 master_runner.cc:187] Error getting master registration for 127.20.202.252:46631: Network error: Client connection negotiation failed: client connection to 127.20.202.252:46631: connect: Connection refused (error 111),
I20260806 07:51:03.074805 5106 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:03.093868 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:46631
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:03.459694 5195 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:03.460088 5195 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:03.460246 5195 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:03.468556 5195 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:03.468729 5195 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:03.468860 5195 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:03.468935 5195 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:03.481031 5195 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:03.482822 5195 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:03.484323 5195 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:03.492568 5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:03.492878 5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:03.492909 5201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:03.494042 5195 server_base.cc:1061] running on GCE node
I20260806 07:51:03.494774 5195 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:03.494961 5195 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:03.495059 5195 hybrid_clock.cc:648] HybridClock initialized: now 1786002663495019 us; error 0 us; skew 500 ppm
I20260806 07:51:03.495412 5195 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:03 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002663, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:03.499058 5195 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:51:03.500979 5195 webserver.cc:533] Webserver started at http://127.20.202.252:45227/ using document root <none> and password file <none>
I20260806 07:51:03.501747 5195 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:03.501860 5195 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:03.507682 5195 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:51:03.511093 5209 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:03.512560 5195 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:03.512744 5195 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "f25f80bac722409a92ecca195df846d9"
format_stamp: "Formatted at 2026-08-06 07:50:54 on dist-test-slave-g350"
I20260806 07:51:03.513501 5195 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:03.549778 5195 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:03.554137 5195 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:03.554416 5195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:03.575209 5262 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:46631 every 8 connection(s)
I20260806 07:51:03.575233 5195 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:46631
I20260806 07:51:03.576864 5195 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:51:03.581007 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5195
I20260806 07:51:03.581238 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:51:03.585474 5263 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:03.592113 5263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Bootstrap starting.
Aug 06 07:51:03 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:03.614697 5263 log.cc:826] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Log is configured to *not* fsync() on all Append() calls
Aug 06 07:51:03 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:03.625907 5263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:03.626500 5263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: Bootstrap complete.
I20260806 07:51:03.635135 5263 raft_consensus.cc:359] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.635969 5263 raft_consensus.cc:740] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25f80bac722409a92ecca195df846d9, State: Initialized, Role: FOLLOWER
I20260806 07:51:03.636634 5263 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.637580 5270 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:03.638042 5270 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:03.639029 5263 sys_catalog.cc:565] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 07:51:03 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:03.652779 5281 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:51:03.652882 5281 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: loading cluster ID for follower catalog manager: success
I20260806 07:51:03.658026 5281 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: acquiring CA information for follower catalog manager: success
I20260806 07:51:03.660336 5281 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:03.688352 5195 master_runner.cc:264] No leader master found from master f25f80bac722409a92ecca195df846d9
I20260806 07:51:03.688522 5195 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:03.714627 5284 raft_consensus.cc:493] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:51:03.715029 5284 raft_consensus.cc:515] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.717571 5284 leader_election.cc:290] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce9b20436581491dab395958d70e4796 (127.20.202.253:40617), f25f80bac722409a92ecca195df846d9 (127.20.202.252:46631)
I20260806 07:51:03.728056 5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34335818920843458fa8203dca58c94f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce9b20436581491dab395958d70e4796" is_pre_election: true
I20260806 07:51:03.728430 5238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34335818920843458fa8203dca58c94f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f25f80bac722409a92ecca195df846d9" is_pre_election: true
I20260806 07:51:03.728744 5147 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34335818920843458fa8203dca58c94f in term 1.
I20260806 07:51:03.729116 5238 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34335818920843458fa8203dca58c94f in term 1.
I20260806 07:51:03.729499 5034 leader_election.cc:304] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34335818920843458fa8203dca58c94f, ce9b20436581491dab395958d70e4796; no voters:
I20260806 07:51:03.730010 5284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:51:03.730188 5284 raft_consensus.cc:493] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:51:03.730329 5284 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:03.734429 5284 raft_consensus.cc:515] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.735242 5284 leader_election.cc:290] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [CANDIDATE]: Term 2 election: Requested vote from peers ce9b20436581491dab395958d70e4796 (127.20.202.253:40617), f25f80bac722409a92ecca195df846d9 (127.20.202.252:46631)
I20260806 07:51:03.735772 5147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34335818920843458fa8203dca58c94f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce9b20436581491dab395958d70e4796"
I20260806 07:51:03.735929 5238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34335818920843458fa8203dca58c94f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f25f80bac722409a92ecca195df846d9"
I20260806 07:51:03.736023 5147 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:03.736162 5238 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:03.742220 5147 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34335818920843458fa8203dca58c94f in term 2.
I20260806 07:51:03.742229 5238 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34335818920843458fa8203dca58c94f in term 2.
I20260806 07:51:03.744208 5033 leader_election.cc:304] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34335818920843458fa8203dca58c94f, f25f80bac722409a92ecca195df846d9; no voters:
I20260806 07:51:03.744616 5284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:51:03.745220 5284 raft_consensus.cc:697] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 LEADER]: Becoming Leader. State: Replica: 34335818920843458fa8203dca58c94f, State: Running, Role: LEADER
I20260806 07:51:03.745944 5284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:03.749487 5287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34335818920843458fa8203dca58c94f. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:03.749816 5287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:03.750634 5289 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:03.752621 5289 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:03.753423 5289 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:51:03.753506 5289 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:03.754688 5289 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:03.755501 5289 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Loaded TSK: 0
I20260806 07:51:03.756464 5289 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:03.882166 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:04.159422 5147 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Refusing update from remote peer 34335818920843458fa8203dca58c94f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:51:04.160665 5287 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:51:04.164413 5295 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34335818920843458fa8203dca58c94f. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:04.164872 5295 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:04.167093 5287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:04.167340 5287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:04.168246 5238 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 2 FOLLOWER]: Refusing update from remote peer 34335818920843458fa8203dca58c94f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:51:04.169361 5287 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:51:04.178474 5295 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:04.178769 5295 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:04.182277 5303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:04.182595 5303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:04.182627 5300 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34335818920843458fa8203dca58c94f. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:04.182978 5300 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:51:04.242714 5294 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:04.243007 5294 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:04.243194 5294 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:04.251595 5294 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:04.251820 5294 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:04.262475 5294 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:04.264108 5294 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:04.265468 5294 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:04.275312 5309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:04.275345 5308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:04.276089 5311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:04.276705 5294 server_base.cc:1061] running on GCE node
I20260806 07:51:04.277287 5294 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:04.277485 5294 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:04.277588 5294 hybrid_clock.cc:648] HybridClock initialized: now 1786002664277551 us; error 0 us; skew 500 ppm
I20260806 07:51:04.277933 5294 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:04 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:04.281454 5294 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:04.283042 5294 webserver.cc:533] Webserver started at http://127.20.202.193:43433/ using document root <none> and password file <none>
I20260806 07:51:04.283887 5294 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:04.284022 5294 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:04.289285 5294 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:04.291973 5317 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:04.293403 5294 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:51:04.293653 5294 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:51:04.294415 5294 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:04.307559 5294 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:04.311352 5294 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:04.311617 5294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:04.312551 5294 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:04.314162 5294 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:04.314272 5294 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:04.314399 5294 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:04.314502 5294 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:04.349355 5294 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33913
I20260806 07:51:04.349444 5430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33913 every 8 connection(s)
I20260806 07:51:04.351150 5294 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:04.359506 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5294
I20260806 07:51:04.359715 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:51:04 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:04 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:04.378268 5431 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:51:04.378320 5433 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:51:04.378747 5433 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:04.379778 5433 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:51:04.378746 5431 heartbeater.cc:461] Registering TS with master...
Aug 06 07:51:04 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:04.381206 5431 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:51:04.382536 5136 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:33913)
I20260806 07:51:04.383865 5228 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:33913)
I20260806 07:51:04.388955 5432 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:51:04.389127 5432 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:04.389561 5432 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:51:04.391345 5047 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:33913)
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:04.393851 5047 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:35845
W20260806 07:51:04.418991 5079 messenger.cc:194] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.202.254:46631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:04.473084 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:04.559607 5299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "34335818920843458fa8203dca58c94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:04.559962 5299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:04.581255 5259 messenger.cc:194] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.202.252:46631
W20260806 07:51:04.815963 5445 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:04.816305 5445 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:04.816486 5445 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:04.824916 5445 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:04.825162 5445 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:51:04.835825 5445 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:04.837495 5445 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:04.839020 5445 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:04.850402 5451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:04.850937 5450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:04.851426 5453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:04.851975 5445 server_base.cc:1061] running on GCE node
I20260806 07:51:04.852571 5445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:04.852752 5445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:04.852854 5445 hybrid_clock.cc:648] HybridClock initialized: now 1786002664852819 us; error 0 us; skew 500 ppm
I20260806 07:51:04.853158 5445 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:04 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:04.856969 5445 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:51:04.859138 5445 webserver.cc:533] Webserver started at http://127.20.202.194:33307/ using document root <none> and password file <none>
I20260806 07:51:04.859877 5445 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:04.859983 5445 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:04.865389 5445 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:04.868144 5460 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:04.869549 5445 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:51:04.869756 5445 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "dba3c45e12cd4dc8b72b29b186a5137d"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:51:04.870500 5445 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:04.900725 5445 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:04.905112 5445 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:04.905429 5445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:04.906577 5445 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:04.908241 5445 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:04.908368 5445 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:04.908504 5445 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:04.908592 5445 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:04.944818 5445 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:46183
I20260806 07:51:04.944908 5575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:46183 every 8 connection(s)
I20260806 07:51:04.946465 5445 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:51:04.948170 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5445
I20260806 07:51:04.948402 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:05.018074 5577 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:05.018536 5577 heartbeater.cc:461] Registering TS with master...
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002664, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:05.019989 5577 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:51:05.022267 5047 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:46183)
I20260806 07:51:05.023748 5047 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:59513
I20260806 07:51:05.025532 5578 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:51:05.025714 5578 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:05.026535 5578 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:51:05.027617 5136 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:46183)
I20260806 07:51:05.029414 5576 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:51:05.029568 5576 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:05.029898 5576 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:51:05.031065 5228 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:46183)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:05.100957 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:05.396965 5432 heartbeater.cc:499] Master 127.20.202.254:46631 was elected leader, sending a full tablet report...
W20260806 07:51:05.444243 5590 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:05.444607 5590 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:05.444764 5590 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:05.453087 5590 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:05.453336 5590 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:51:05.464288 5590 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:05.466420 5590 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:05.467929 5590 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:05.479393 5595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:05.479431 5596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:05.482827 5598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:05.483474 5590 server_base.cc:1061] running on GCE node
I20260806 07:51:05.484051 5590 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:05.484237 5590 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:05.484338 5590 hybrid_clock.cc:648] HybridClock initialized: now 1786002665484314 us; error 0 us; skew 500 ppm
I20260806 07:51:05.484660 5590 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002665, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:05.488282 5590 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:51:05.489892 5590 webserver.cc:533] Webserver started at http://127.20.202.195:34791/ using document root <none> and password file <none>
I20260806 07:51:05.490581 5590 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:05.490685 5590 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:05.496093 5590 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 07:51:05.498827 5604 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:05.500379 5590 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:05.500564 5590 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "c4dabe24cfd342eb8cf974dd42356343"
format_stamp: "Formatted at 2026-08-06 07:50:56 on dist-test-slave-g350"
I20260806 07:51:05.501278 5590 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:05.528599 5590 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:05.532449 5590 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:05.532730 5590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:05.533767 5590 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:05.535470 5590 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:05.535558 5590 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:05.535727 5590 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:05.535816 5590 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:05.572171 5590 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:36639
I20260806 07:51:05.572214 5718 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:36639 every 8 connection(s)
I20260806 07:51:05.573895 5590 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:51:05.576164 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5590
I20260806 07:51:05.576359 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002665, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002665, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:05 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002665, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:05.617314 5720 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:51:05.617800 5720 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:05.618976 5720 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:51:05.620311 5723 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:51:05.620468 5723 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:05.620712 5047 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:36639)
I20260806 07:51:05.621021 5723 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:51:05.621976 5047 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:41413
I20260806 07:51:05.622135 5136 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:36639)
I20260806 07:51:05.622216 5719 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:51:05.622330 5719 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:05.622844 5719 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:51:05.623936 5228 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:36639)
I20260806 07:51:05.634920 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:05.642314 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:05.642477 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5016
I20260806 07:51:05.643352 5078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:05.756392 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5016
W20260806 07:51:05.779703 5213 connection.cc:573] server connection from 127.0.0.1:37408 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:51:05.780867 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--webserver_interface=127.20.202.254
--webserver_port=33241
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:06.027786 5577 heartbeater.cc:499] Master 127.20.202.254:46631 was elected leader, sending a full tablet report...
W20260806 07:51:06.029688 5577 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:46631 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:46631: connect: Connection refused (error 111)
W20260806 07:51:06.126228 5730 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:06.126597 5730 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:06.126752 5730 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:06.135303 5730 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:06.135442 5730 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:06.135566 5730 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:06.135668 5730 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:06.146761 5730 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=33241
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:06.148447 5730 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:06.149957 5730 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:06.158252 5737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:06.158280 5736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:06.159314 5739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:06.159582 5730 server_base.cc:1061] running on GCE node
I20260806 07:51:06.160367 5730 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:06.160504 5730 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:06.160570 5730 hybrid_clock.cc:648] HybridClock initialized: now 1786002666160554 us; error 0 us; skew 500 ppm
I20260806 07:51:06.160861 5730 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:06 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002666, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:06.164724 5730 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:06.166791 5730 webserver.cc:533] Webserver started at http://127.20.202.254:33241/ using document root <none> and password file <none>
I20260806 07:51:06.167505 5730 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:06.167629 5730 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:06.172772 5730 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 07:51:06.175707 5745 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:06.177449 5730 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:06.177711 5730 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "34335818920843458fa8203dca58c94f"
format_stamp: "Formatted at 2026-08-06 07:50:53 on dist-test-slave-g350"
I20260806 07:51:06.178509 5730 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:06.195585 5730 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:06.199558 5730 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:06.199838 5730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:06.221225 5730 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:46631
I20260806 07:51:06.221278 5797 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:46631 every 8 connection(s)
I20260806 07:51:06.222952 5730 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:06.225107 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5730
I20260806 07:51:06.225512 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5294
I20260806 07:51:06.231711 5798 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:06.235356 5798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap starting.
Aug 06 07:51:06 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:51:06.244326 5426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:06.254401 5798 log.cc:826] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:06.263288 5798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:06.263825 5798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: Bootstrap complete.
I20260806 07:51:06.270604 5798 raft_consensus.cc:359] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:06.271302 5798 raft_consensus.cc:740] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34335818920843458fa8203dca58c94f, State: Initialized, Role: FOLLOWER
I20260806 07:51:06.272099 5798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:06.272979 5804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:06.273285 5804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:06.273964 5798 sys_catalog.cc:565] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:06.287585 5815 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:51:06.287742 5815 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: loading cluster ID for follower catalog manager: success
I20260806 07:51:06.291738 5815 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: acquiring CA information for follower catalog manager: success
I20260806 07:51:06.294325 5815 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:06.331326 5730 master_runner.cc:264] No leader master found from master 34335818920843458fa8203dca58c94f
I20260806 07:51:06.331480 5730 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:06.346668 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5294
I20260806 07:51:06.366472 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33913
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--webserver_port=43433
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:06.625412 5720 heartbeater.cc:499] Master 127.20.202.254:46631 was elected leader, sending a full tablet report...
I20260806 07:51:06.637403 5762 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c4dabe24cfd342eb8cf974dd42356343" instance_seqno: 1786002665563642) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:43249; Asking this server to re-register.
I20260806 07:51:06.638463 5720 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:06.640424 5762 ts_manager.cc:194] Registered new tserver with Master: c4dabe24cfd342eb8cf974dd42356343 (127.20.202.195:36639)
W20260806 07:51:06.712853 5821 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:06.713177 5821 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:06.713332 5821 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:06.721976 5821 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:06.722195 5821 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:06.732896 5821 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43433
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:06.734592 5821 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:06.736022 5821 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:51:06.739262 5826 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34335818920843458fa8203dca58c94f)
I20260806 07:51:06.739516 5826 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:06.741564 5826 leader_election.cc:290] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34335818920843458fa8203dca58c94f (127.20.202.254:46631), f25f80bac722409a92ecca195df846d9 (127.20.202.252:46631)
W20260806 07:51:06.742233 5262 acceptor_pool.cc:311] unable to collect diagnostics on RPC socket 127.20.202.252:46631: Invalid argument: socket address is ambiguous
W20260806 07:51:06.746599 5830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:06.747355 5831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:06.751844 5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34335818920843458fa8203dca58c94f" is_pre_election: true
I20260806 07:51:06.752447 5772 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9b20436581491dab395958d70e4796 in term 2.
I20260806 07:51:06.752905 5238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f25f80bac722409a92ecca195df846d9" is_pre_election: true
I20260806 07:51:06.753211 5124 leader_election.cc:304] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34335818920843458fa8203dca58c94f, ce9b20436581491dab395958d70e4796; no voters:
I20260806 07:51:06.753304 5821 server_base.cc:1061] running on GCE node
I20260806 07:51:06.753746 5826 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:51:06.753888 5826 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34335818920843458fa8203dca58c94f)
I20260806 07:51:06.754055 5826 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 2 FOLLOWER]: Advancing to term 3
W20260806 07:51:06.754225 5833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:06.754954 5821 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:06.755177 5821 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:06.755281 5821 hybrid_clock.cc:648] HybridClock initialized: now 1786002666755259 us; error 0 us; skew 500 ppm
I20260806 07:51:06.755631 5821 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:51:06.756479 5826 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:06.757200 5826 leader_election.cc:290] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 3 election: Requested vote from peers 34335818920843458fa8203dca58c94f (127.20.202.254:46631), f25f80bac722409a92ecca195df846d9 (127.20.202.252:46631)
I20260806 07:51:06.757714 5238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f25f80bac722409a92ecca195df846d9"
I20260806 07:51:06.757905 5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce9b20436581491dab395958d70e4796" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34335818920843458fa8203dca58c94f"
I20260806 07:51:06.758107 5772 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 2 FOLLOWER]: Advancing to term 3
Aug 06 07:51:06 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002666, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:06.760070 5821 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:06.762125 5772 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9b20436581491dab395958d70e4796 in term 3.
I20260806 07:51:06.762605 5124 leader_election.cc:304] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34335818920843458fa8203dca58c94f, ce9b20436581491dab395958d70e4796; no voters: f25f80bac722409a92ecca195df846d9
I20260806 07:51:06.762974 5826 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:51:06.763481 5826 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [term 3 LEADER]: Becoming Leader. State: Replica: ce9b20436581491dab395958d70e4796, State: Running, Role: LEADER
I20260806 07:51:06.763535 5821 webserver.cc:533] Webserver started at http://127.20.202.193:43433/ using document root <none> and password file <none>
I20260806 07:51:06.764058 5826 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } }
I20260806 07:51:06.764528 5821 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:06.764663 5821 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:06.766137 5838 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:06.766459 5838 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:06.767210 5840 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:06.769101 5840 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:06.770243 5840 catalog_manager.cc:1295] Loaded cluster ID: 5d007c990f624c8d8d378828f7cbfe10
I20260806 07:51:06.770355 5840 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:06.771076 5821 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:51:06.771540 5840 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:06.772343 5840 catalog_manager.cc:6081] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796: Loaded TSK: 0
I20260806 07:51:06.773329 5840 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:51:06.774320 5842 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:06.775955 5821 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:06.776214 5821 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:51:06.777010 5821 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:06.793678 5821 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:06.797720 5821 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:06.797999 5821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:06.798967 5821 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:06.800536 5821 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:06.800661 5821 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:06.800791 5821 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:06.800895 5821 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:06.835844 5821 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33913
I20260806 07:51:06.835937 5955 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33913 every 8 connection(s)
I20260806 07:51:06.837515 5821 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:06.842959 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5821
Aug 06 07:51:06 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002666, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:06 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002666, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:06 dist-test-slave-g350 krb5kdc[5002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002666, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:06.870096 5956 heartbeater.cc:344] Connected to a master server at 127.20.202.252:46631
I20260806 07:51:06.870280 5958 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40617
I20260806 07:51:06.870519 5958 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:06.870535 5956 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:06.870838 5957 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:51:06.870985 5957 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:06.871649 5956 heartbeater.cc:507] Master 127.20.202.252:46631 requested a full tablet report, sending...
I20260806 07:51:06.871670 5958 heartbeater.cc:507] Master 127.20.202.253:40617 requested a full tablet report, sending...
I20260806 07:51:06.872363 5957 heartbeater.cc:507] Master 127.20.202.254:46631 requested a full tablet report, sending...
I20260806 07:51:06.873871 5762 ts_manager.cc:194] Registered new tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:33913)
I20260806 07:51:06.873963 5228 ts_manager.cc:194] Re-registered known tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:33913)
I20260806 07:51:06.873968 5136 ts_manager.cc:194] Re-registered known tserver with Master: d35aba5a0456484f8b8647a2aa097a0d (127.20.202.193:33913)
I20260806 07:51:06.875938 5136 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:43637
I20260806 07:51:06.892741 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5821
W20260806 07:51:06.893944 5967 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:06.890353 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.20.202.193:33913)
0806 07:51:06.891110 (+ 757us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:06.891117 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:06.891147 (+ 30us) server_negotiation.cc:408] Connection header received
0806 07:51:06.891210 (+ 63us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:06.891217 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:06.891423 (+ 206us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:06.891594 (+ 171us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:06.892423 (+ 829us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48206: BlockingRecv error: recv got EOF from 127.0.0.1:48206 (error 108)
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":421,"threads_started":1}
W20260806 07:51:06.895159 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5821: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33913: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:51:06.895284 5967 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:06.893755 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.20.202.193:33913)
0806 07:51:06.894244 (+ 489us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:06.894249 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:06.894270 (+ 21us) server_negotiation.cc:408] Connection header received
0806 07:51:06.894325 (+ 55us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:06.894329 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:06.894403 (+ 74us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:06.894508 (+ 105us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:06.895052 (+ 544us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48208: BlockingRecv error: recv got EOF from 127.0.0.1:48208 (error 108)
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":611,"threads_started":1}
I20260806 07:51:06.895364 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5821
I20260806 07:51:06.914412 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33913
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--webserver_port=43433
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:07.038657 5577 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46631
I20260806 07:51:07.038883 5577 heartbeater.cc:499] Master 127.20.202.254:46631 was elected leader, sending a full tablet report...
I20260806 07:51:07.039816 5762 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dba3c45e12cd4dc8b72b29b186a5137d" instance_seqno: 1786002664935471) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:32817; Asking this server to re-register.
I20260806 07:51:07.040699 5577 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:07.042006 5762 ts_manager.cc:194] Registered new tserver with Master: dba3c45e12cd4dc8b72b29b186a5137d (127.20.202.194:46183)
I20260806 07:51:07.162485 5238 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:07.164824 5238 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [term 3 FOLLOWER]: Refusing update from remote peer ce9b20436581491dab395958d70e4796: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:51:07.165673 5838 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:51:07.168069 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:07.168462 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:07.170269 5838 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:07.170521 5838 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:07.171531 5772 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [term 3 FOLLOWER]: Refusing update from remote peer ce9b20436581491dab395958d70e4796: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:51:07.172230 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:07.172425 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25f80bac722409a92ecca195df846d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:07.172478 5826 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:51:07.176338 5978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:07.176672 5978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:07.177433 5975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9b20436581491dab395958d70e4796. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:07.177748 5975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34335818920843458fa8203dca58c94f [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:51:07.227488 5794 messenger.cc:194] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.202.254:46631
W20260806 07:51:07.258860 5969 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:07.259215 5969 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:07.259371 5969 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:51:07.259465 5969 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:07.267867 5969 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:07.268131 5969 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:07.279647 5969 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43433
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:07.281319 5969 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:07.282784 5969 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:07.293452 5983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:07.293668 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:07.293686 5986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:07.294348 5969 server_base.cc:1061] running on GCE node
I20260806 07:51:07.294857 5969 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:07.295037 5969 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:07.295151 5969 hybrid_clock.cc:648] HybridClock initialized: now 1786002667295112 us; error 0 us; skew 500 ppm
I20260806 07:51:07.295449 5969 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:07 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002667, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:07.299229 5969 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:07.300871 5969 webserver.cc:533] Webserver started at http://127.20.202.193:43433/ using document root <none> and password file <none>
I20260806 07:51:07.301735 5969 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:07.301862 5969 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:07.307662 5969 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:51:07.310446 5992 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:07.311839 5969 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:07.312012 5969 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:51:07.312769 5969 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:07.340270 5969 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:07.344664 5969 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:07.344964 5969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:07.345943 5969 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:07.347573 5969 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:07.347645 5969 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:07.347815 5969 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:07.347872 5969 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:51:07.348272 5969 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:51:07.348362 5969 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:51:07.348444 5969 fault_injection.cc:59] injecting fault for kudu (pid 5969): FLAGS_fault_before_start (process will exit)
I20260806 07:51:07.369247 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33913
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--webserver_port=43433
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:07.641453 5826 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ce9b20436581491dab395958d70e4796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34335818920843458fa8203dca58c94f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46631 } } peers { permanent_uuid: "ce9b20436581491dab395958d70e4796" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40617 } } peers { permanent_uuid: "f25f80bac722409a92ecca195df846d9" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 46631 } } }
I20260806 07:51:07.641784 5826 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9b20436581491dab395958d70e4796 [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:07.727949 6000 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:07.728296 6000 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:07.728474 6000 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:07.736897 6000 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:07.737151 6000 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:07.748024 6000 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43433
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46631,127.20.202.253:40617,127.20.202.252:46631
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:07.749794 6000 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:07.751219 6000 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:07.761696 6006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:07.761682 6007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:07.762257 6009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:07.763028 6000 server_base.cc:1061] running on GCE node
I20260806 07:51:07.763583 6000 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:51:07.763751 6000 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:51:07.763841 6000 hybrid_clock.cc:648] HybridClock initialized: now 1786002667763823 us; error 0 us; skew 500 ppm
I20260806 07:51:07.764156 6000 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:07 dist-test-slave-g350 krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002667, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:07.767824 6000 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:07.769475 6000 webserver.cc:533] Webserver started at http://127.20.202.193:43433/ using document root <none> and password file <none>
I20260806 07:51:07.770262 6000 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:07.770359 6000 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:07.776262 6000 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 07:51:07.779188 6015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:07.780648 6000 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:07.780835 6000 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "d35aba5a0456484f8b8647a2aa097a0d"
format_stamp: "Formatted at 2026-08-06 07:50:55 on dist-test-slave-g350"
I20260806 07:51:07.781566 6000 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:07.803727 6000 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:07.811751 6000 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:07.812052 6000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:07.813130 6000 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:07.814859 6000 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:07.814981 6000 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:07.815115 6000 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:07.815207 6000 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:07.851255 6000 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33913
I20260806 07:51:07.851347 6128 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33913 every 8 connection(s)
I20260806 07:51:07.852927 6000 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:07.853883 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6000
I20260806 07:51:07.854315 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6000
W20260806 07:51:07.871429 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6000: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33913: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:07.871670 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6000
I20260806 07:51:07.890182 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5445
W20260806 07:51:07.890964 6134 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:07.857195 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:47915 (local address 127.20.202.252:46631)
0806 07:51:07.865343 (+ 8148us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:07.865352 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:07.865384 (+ 32us) server_negotiation.cc:408] Connection header received
0806 07:51:07.865474 (+ 90us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:07.865483 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:07.865611 (+ 128us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:07.865775 (+ 164us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:07.869106 (+ 3331us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:07.870970 (+ 1864us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:07.890008 (+ 19038us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:47915: BlockingRecv error: recv got EOF from 127.20.202.193:47915 (error 108)
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":243,"threads_started":1}
W20260806 07:51:07.890967 6135 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:07.857756 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:46949 (local address 127.20.202.253:40617)
0806 07:51:07.875264 (+ 17508us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:07.875268 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:07.889875 (+ 14607us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:46949: BlockingRecv error: recv got EOF from 127.20.202.193:46949 (error 108)
Metrics: {"server-negotiator.queue_time_us":17386,"thread_start_us":17276,"threads_started":1}
W20260806 07:51:07.893074 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5445: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:46183: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:07.893230 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5445
I20260806 07:51:07.911448 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5590
W20260806 07:51:07.914666 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5590: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:36639: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:07.914856 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5590
I20260806 07:51:07.932273 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5730
W20260806 07:51:07.934607 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5730: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:46631: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:07.934785 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5730
I20260806 07:51:07.959391 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5106
I20260806 07:51:07.960628 5167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:08.074510 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5106
I20260806 07:51:08.094734 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5195
I20260806 07:51:08.095885 5258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:08.204317 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5195
[ OK ] ExternalMiniClusterTest.TestBasicOperation/6 (17767 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/7
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:51:08 dist-test-slave-g350 krb5kdc[6146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:51:08 dist-test-slave-g350 krb5kdc[6146](info): set up 2 sockets
Aug 06 07:51:08 dist-test-slave-g350 krb5kdc[6146](info): commencing operation
krb5kdc: starting...
W20260806 07:51:10.293377 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.051s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:51:10 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:51:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:10.582237 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:10.922122 6162 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:10.922425 6162 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:10.922600 6162 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:10.930866 6162 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:10.930958 6162 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:10.931068 6162 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:10.931150 6162 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:10.941977 6162 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:10.943693 6162 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:10.945080 6162 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:10.953465 6168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:10.953462 6167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:10.954200 6170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:10.955135 6162 server_base.cc:1061] running on GCE node
I20260806 07:51:10.955902 6162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:10.957309 6162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:10.958534 6162 hybrid_clock.cc:648] HybridClock initialized: now 1786002670958454 us; error 80 us; skew 500 ppm
I20260806 07:51:10.958887 6162 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:10 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:10.962250 6162 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:10.963976 6162 webserver.cc:533] Webserver started at http://127.20.202.254:43543/ using document root <none> and password file <none>
I20260806 07:51:10.964745 6162 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:10.964854 6162 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:10.965171 6162 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:10.967752 6162 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "9f3d2d52491f4b38b115d58397720f4c"
format_stamp: "Formatted at 2026-08-06 07:51:10 on dist-test-slave-g350"
I20260806 07:51:10.968578 6162 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "9f3d2d52491f4b38b115d58397720f4c"
format_stamp: "Formatted at 2026-08-06 07:51:10 on dist-test-slave-g350"
I20260806 07:51:10.974519 6162 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 07:51:10.978101 6177 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:10.979506 6162 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:51:10.979698 6162 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "9f3d2d52491f4b38b115d58397720f4c"
format_stamp: "Formatted at 2026-08-06 07:51:10 on dist-test-slave-g350"
I20260806 07:51:10.979965 6162 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:11.001883 6162 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:11.006194 6162 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:11.006505 6162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:11.029033 6162 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40295
I20260806 07:51:11.029059 6229 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40295 every 8 connection(s)
I20260806 07:51:11.030697 6162 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:11.035565 6230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:51:11.035678 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6162
I20260806 07:51:11.035898 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:51:11.045045 6230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:11 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
I20260806 07:51:11.070354 6230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:51:11.071780 6179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111)
W20260806 07:51:11.073559 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37375: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
I20260806 07:51:11.122967 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } attempt: 1
W20260806 07:51:11.125025 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37375: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:11.141243 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37375
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:11.193265 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } attempt: 2
W20260806 07:51:11.195542 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37375: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111)
I20260806 07:51:11.286834 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } attempt: 3
W20260806 07:51:11.289036 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37375: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111)
I20260806 07:51:11.432328 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } attempt: 4
W20260806 07:51:11.434757 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37375: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111)
W20260806 07:51:11.491240 6239 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:11.491611 6239 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:11.491767 6239 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:11.500360 6239 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:11.500510 6239 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:11.500633 6239 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:11.500723 6239 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:11.511782 6239 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:11.513535 6239 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:11.515136 6239 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:11.523690 6245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:11.523656 6244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:11.524636 6247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:11.525184 6239 server_base.cc:1061] running on GCE node
I20260806 07:51:11.526028 6239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:11.527412 6239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:11.528672 6239 hybrid_clock.cc:648] HybridClock initialized: now 1786002671528631 us; error 44 us; skew 500 ppm
I20260806 07:51:11.529065 6239 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:11 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002671, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:11.532243 6239 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:51:11.533875 6239 webserver.cc:533] Webserver started at http://127.20.202.253:36289/ using document root <none> and password file <none>
I20260806 07:51:11.534591 6239 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:11.534722 6239 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:11.535089 6239 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:11.538252 6239 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "21536eb957e04c83accf33b3f72ceae3"
format_stamp: "Formatted at 2026-08-06 07:51:11 on dist-test-slave-g350"
I20260806 07:51:11.539291 6239 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "21536eb957e04c83accf33b3f72ceae3"
format_stamp: "Formatted at 2026-08-06 07:51:11 on dist-test-slave-g350"
I20260806 07:51:11.544631 6239 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:51:11.548110 6255 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:11.549751 6239 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:51:11.549892 6239 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "21536eb957e04c83accf33b3f72ceae3"
format_stamp: "Formatted at 2026-08-06 07:51:11 on dist-test-slave-g350"
I20260806 07:51:11.550127 6239 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:11.581084 6239 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:11.584903 6239 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:11.585224 6239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:11.605563 6239 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:37375
I20260806 07:51:11.605636 6306 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:37375 every 8 connection(s)
I20260806 07:51:11.607173 6239 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:51:11.611492 6307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:51:11.616335 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6239
I20260806 07:51:11.616564 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:51:11.617537 6307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:11 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002671, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:11.641487 6307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:51:11 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002671, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:11.657488 6307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:51:11.658876 6259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
W20260806 07:51:11.660686 6307 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:51:11.709985 6307 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 1
W20260806 07:51:11.712229 6307 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:11.724261 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41915
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:11.734031 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } attempt: 5
Aug 06 07:51:11 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:11.748515 6230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:51:11.750456 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
I20260806 07:51:11.780493 6307 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 2
W20260806 07:51:11.782514 6307 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
I20260806 07:51:11.801683 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 1
W20260806 07:51:11.803859 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
I20260806 07:51:11.872167 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 2
I20260806 07:51:11.873735 6307 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 3
W20260806 07:51:11.874502 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
W20260806 07:51:11.875705 6307 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
I20260806 07:51:11.980753 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 3
W20260806 07:51:11.983080 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
I20260806 07:51:12.018934 6307 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 4
W20260806 07:51:12.021163 6307 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
W20260806 07:51:12.069427 6318 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:12.069782 6318 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:12.069942 6318 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:12.078222 6318 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:12.078313 6318 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:12.078437 6318 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:12.078510 6318 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:12.089236 6318 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:12.090969 6318 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:12.092329 6318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:12.100500 6325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:12.100490 6324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:12.101361 6327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:12.101675 6318 server_base.cc:1061] running on GCE node
I20260806 07:51:12.102401 6318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:12.103754 6318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:12.105079 6318 hybrid_clock.cc:648] HybridClock initialized: now 1786002672105003 us; error 68 us; skew 500 ppm
I20260806 07:51:12.105472 6318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:12.108690 6318 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:51:12.110395 6318 webserver.cc:533] Webserver started at http://127.20.202.252:37915/ using document root <none> and password file <none>
I20260806 07:51:12.111131 6318 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:12.111263 6318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:12.111608 6318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:12.114037 6318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "e3d38b17f71a4cf58eb0d96ebc57007d"
format_stamp: "Formatted at 2026-08-06 07:51:12 on dist-test-slave-g350"
I20260806 07:51:12.114836 6318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "e3d38b17f71a4cf58eb0d96ebc57007d"
format_stamp: "Formatted at 2026-08-06 07:51:12 on dist-test-slave-g350"
I20260806 07:51:12.120096 6318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:51:12.123607 6334 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:12.124989 6318 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:12.125168 6318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "e3d38b17f71a4cf58eb0d96ebc57007d"
format_stamp: "Formatted at 2026-08-06 07:51:12 on dist-test-slave-g350"
I20260806 07:51:12.125356 6318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:12.140836 6318 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:12.144622 6318 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:12.144905 6318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:12.160341 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 4
W20260806 07:51:12.162562 6230 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111)
I20260806 07:51:12.166142 6318 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:41915
I20260806 07:51:12.166220 6385 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:41915 every 8 connection(s)
I20260806 07:51:12.167747 6318 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:51:12.168394 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6318
I20260806 07:51:12.168597 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:51:12.173563 6386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:12.179723 6386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:12.201894 6386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:12.215972 6386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002672, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:12.233662 6386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Bootstrap starting.
I20260806 07:51:12.237005 6386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Neither blocks nor log segments found. Creating new log.
I20260806 07:51:12.238193 6386 log.cc:826] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:12.240495 6386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: No bootstrap required, opened a new log
I20260806 07:51:12.246052 6386 raft_consensus.cc:359] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.246307 6386 raft_consensus.cc:385] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:51:12.246402 6386 raft_consensus.cc:740] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d38b17f71a4cf58eb0d96ebc57007d, State: Initialized, Role: FOLLOWER
I20260806 07:51:12.246918 6386 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.247731 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.248016 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:12.248672 6386 sys_catalog.cc:565] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:12.251170 6394 raft_consensus.cc:493] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:51:12.251410 6394 raft_consensus.cc:515] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.253523 6394 leader_election.cc:290] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f3d2d52491f4b38b115d58397720f4c (127.20.202.254:40295), 21536eb957e04c83accf33b3f72ceae3 (127.20.202.253:37375)
W20260806 07:51:12.259181 6406 catalog_manager.cc:1594] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:51:12.259349 6406 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:51:12.257701 6318 master_runner.cc:187] Error getting master registration for 127.20.202.254:40295: OK, instance_id { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" instance_seqno: 1786002671023105 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:51:12.264988 6318 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:12.320386 6307 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 5
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002671, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:12.341810 6307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Bootstrap starting.
I20260806 07:51:12.346222 6307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Neither blocks nor log segments found. Creating new log.
I20260806 07:51:12.349177 6307 log.cc:826] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:12.353260 6307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: No bootstrap required, opened a new log
I20260806 07:51:12.361217 6307 raft_consensus.cc:359] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.361562 6307 raft_consensus.cc:385] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:51:12.361720 6307 raft_consensus.cc:740] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21536eb957e04c83accf33b3f72ceae3, State: Initialized, Role: FOLLOWER
I20260806 07:51:12.362751 6307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.364475 6409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.364928 6409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:12.365307 6307 sys_catalog.cc:565] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:12.365360 6282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21536eb957e04c83accf33b3f72ceae3" is_pre_election: true
I20260806 07:51:12.365896 6282 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d38b17f71a4cf58eb0d96ebc57007d in term 0.
I20260806 07:51:12.366662 6336 leader_election.cc:304] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21536eb957e04c83accf33b3f72ceae3, e3d38b17f71a4cf58eb0d96ebc57007d; no voters:
I20260806 07:51:12.367180 6394 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:51:12.367357 6394 raft_consensus.cc:493] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:51:12.367475 6394 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:51:12.370869 6394 raft_consensus.cc:515] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.371687 6394 leader_election.cc:290] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 1 election: Requested vote from peers 9f3d2d52491f4b38b115d58397720f4c (127.20.202.254:40295), 21536eb957e04c83accf33b3f72ceae3 (127.20.202.253:37375)
I20260806 07:51:12.372268 6282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21536eb957e04c83accf33b3f72ceae3"
I20260806 07:51:12.372478 6282 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:51:12.376210 6282 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d38b17f71a4cf58eb0d96ebc57007d in term 1.
I20260806 07:51:12.376796 6336 leader_election.cc:304] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21536eb957e04c83accf33b3f72ceae3, e3d38b17f71a4cf58eb0d96ebc57007d; no voters:
I20260806 07:51:12.377138 6394 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:51:12.377672 6394 raft_consensus.cc:697] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 LEADER]: Becoming Leader. State: Replica: e3d38b17f71a4cf58eb0d96ebc57007d, State: Running, Role: LEADER
I20260806 07:51:12.378219 6394 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.385211 6414 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.385773 6414 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:12.397058 6340 tablet.cc:2406] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:51:12.399082 6419 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:12.406327 6419 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:12.410714 6239 master_runner.cc:187] Error getting master registration for 127.20.202.254:40295: OK, instance_id { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" instance_seqno: 1786002671023105 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:51:12.420442 6239 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:12.423525 6282 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 1 FOLLOWER]: Refusing update from remote peer e3d38b17f71a4cf58eb0d96ebc57007d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 07:51:12.423525 6423 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:51:12.423871 6423 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:51:12.424685 6414 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:51:12.436546 6409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.436806 6409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:12.440136 6230 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } attempt: 5
I20260806 07:51:12.444728 6425 mvcc.cc:204] Tried to move back new op lower bound from 7315466946234634240 to 7315466946066223104. Current Snapshot: MvccSnapshot[applied={T|T < 7315466946234634240}]
Aug 06 07:51:12 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:12.474802 6230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap starting.
I20260806 07:51:12.484007 6230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Neither blocks nor log segments found. Creating new log.
I20260806 07:51:12.485569 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.486287 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:12.489634 6419 catalog_manager.cc:1383] Generated new cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:12.489771 6419 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:12.492568 6230 log.cc:826] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:12.498930 6409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.499136 6409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:12.507946 6419 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:51:12.508695 6230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: No bootstrap required, opened a new log
I20260806 07:51:12.509913 6419 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:12.525324 6230 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.525715 6230 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:51:12.525884 6230 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3d2d52491f4b38b115d58397720f4c, State: Initialized, Role: FOLLOWER
I20260806 07:51:12.526676 6230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:12.528064 6432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.528638 6432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:12.532882 6419 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Generated new TSK 0
I20260806 07:51:12.534192 6419 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:51:12.546911 6202 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:51:12.549774 6203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3d2d52491f4b38b115d58397720f4c"
I20260806 07:51:12.550066 6204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3d2d52491f4b38b115d58397720f4c" is_pre_election: true
I20260806 07:51:12.553860 6202 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Refusing update from remote peer e3d38b17f71a4cf58eb0d96ebc57007d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:51:12.554874 6394 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:51:12.556260 6230 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:12.598565 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.598938 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:12.599287 6432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:12.599503 6432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
I20260806 07:51:12.681228 6450 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
W20260806 07:51:12.700798 21291 mini_kdc.cc:268] Time spent creating service keytab for kudu/127.20.202.193: real 0.154s user 0.003s sys 0.020s
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:12.753196 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:13.080595 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:13.080946 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:13.122301 6453 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:13.122681 6453 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:13.122841 6453 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:13.131637 6453 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:13.131903 6453 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:13.143184 6453 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:13.144937 6453 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:13.146469 6453 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:13.158245 6461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:13.158609 6462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:13.160204 6464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:13.160442 6453 server_base.cc:1061] running on GCE node
I20260806 07:51:13.161114 6453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:13.162225 6453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:13.163491 6453 hybrid_clock.cc:648] HybridClock initialized: now 1786002673163446 us; error 47 us; skew 500 ppm
I20260806 07:51:13.163828 6453 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:13.171984 6453 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:13.173897 6453 webserver.cc:533] Webserver started at http://127.20.202.193:40911/ using document root <none> and password file <none>
I20260806 07:51:13.174701 6453 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:13.174850 6453 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:13.175240 6453 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:13.177901 6453 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:13.178828 6453 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:13.184785 6453 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 07:51:13.188334 6471 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:13.189815 6453 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:13.189996 6453 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:13.190199 6453 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:13.215633 6453 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:13.222780 6453 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:13.223062 6453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:13.224009 6453 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:13.225759 6453 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:13.225836 6453 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:13.225932 6453 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:13.225978 6453 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:13.262215 6453 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41749
I20260806 07:51:13.262253 6586 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41749 every 8 connection(s)
I20260806 07:51:13.263759 6453 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:13.272166 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6453
I20260806 07:51:13.272399 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.194@KRBTEST.COM" created.
I20260806 07:51:13.336668 6589 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:13.336761 6588 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:13.337131 6588 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:13.337127 6589 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:13.337829 6587 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:13.337954 6587 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:13.338877 6588 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:13.339056 6587 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:13.339977 6589 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:13.342015 6351 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:41749)
I20260806 07:51:13.342033 6194 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:41749)
I20260806 07:51:13.342222 6272 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:41749)
I20260806 07:51:13.344013 6351 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:50749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:13.424945 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:13.426082 6423 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:13.426162 6423 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: loading cluster ID for follower catalog manager: success
I20260806 07:51:13.428795 6423 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: acquiring CA information for follower catalog manager: success
I20260806 07:51:13.430878 6423 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:13.686744 6450 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:13.686898 6450 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: loading cluster ID for follower catalog manager: success
I20260806 07:51:13.690871 6450 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: acquiring CA information for follower catalog manager: success
I20260806 07:51:13.693463 6450 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:51:13.768258 6601 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:13.768589 6601 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:13.768729 6601 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:13.777179 6601 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:13.777401 6601 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:51:13.788344 6601 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:13.790097 6601 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:13.791460 6601 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:13.801232 6607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:13.801724 6608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:13.803099 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:13.804186 6601 server_base.cc:1061] running on GCE node
I20260806 07:51:13.804760 6601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:13.805727 6601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:13.807001 6601 hybrid_clock.cc:648] HybridClock initialized: now 1786002673806939 us; error 46 us; skew 500 ppm
I20260806 07:51:13.807318 6601 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:13.811137 6601 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:51:13.813153 6601 webserver.cc:533] Webserver started at http://127.20.202.194:39629/ using document root <none> and password file <none>
I20260806 07:51:13.814149 6601 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:13.814278 6601 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:13.814692 6601 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:13.818039 6601 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "09e2098fd8404ca0b1ca44a8b18c02ab"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:13.819123 6601 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "09e2098fd8404ca0b1ca44a8b18c02ab"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:13.826113 6601 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260806 07:51:13.830428 6617 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:13.832039 6601 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:51:13.832216 6601 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "09e2098fd8404ca0b1ca44a8b18c02ab"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:13.832425 6601 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:13.858253 6601 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:13.862427 6601 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:13.862723 6601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:13.863790 6601 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:13.865410 6601 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:13.865480 6601 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:13.865638 6601 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:13.865688 6601 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 07:51:13.904762 6601 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:42307
I20260806 07:51:13.904800 6731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:42307 every 8 connection(s)
I20260806 07:51:13.906276 6601 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:51:13.910976 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6601
I20260806 07:51:13.911249 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
I20260806 07:51:13.967326 6734 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:13.967779 6734 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:13.968961 6734 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
Aug 06 07:51:13 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002673, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.195@KRBTEST.COM" created.
I20260806 07:51:13.975507 6272 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:42307)
I20260806 07:51:13.977283 6732 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:13.977424 6732 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:13.978055 6732 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:13.979223 6351 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:42307)
I20260806 07:51:13.980376 6351 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:39469
I20260806 07:51:13.982038 6733 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:13.982223 6733 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:13.982533 6733 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:13.983702 6194 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:42307)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:14.077293 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:14.348013 6587 heartbeater.cc:499] Master 127.20.202.252:41915 was elected leader, sending a full tablet report...
W20260806 07:51:14.422097 6746 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:14.422469 6746 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:14.422612 6746 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:14.431294 6746 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:14.431528 6746 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:51:14.442250 6746 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:14.443917 6746 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:14.445282 6746 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:14.455381 6752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:14.455348 6751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:14.456684 6754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:14.457191 6746 server_base.cc:1061] running on GCE node
I20260806 07:51:14.457860 6746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:14.458834 6746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:14.460058 6746 hybrid_clock.cc:648] HybridClock initialized: now 1786002674459987 us; error 62 us; skew 500 ppm
I20260806 07:51:14.460377 6746 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:14 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:14.463717 6746 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:51:14.465387 6746 webserver.cc:533] Webserver started at http://127.20.202.195:34595/ using document root <none> and password file <none>
I20260806 07:51:14.466158 6746 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:14.466254 6746 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:14.466542 6746 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:14.468951 6746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "5292d011bd654339bed35e282ee662d1"
format_stamp: "Formatted at 2026-08-06 07:51:14 on dist-test-slave-g350"
I20260806 07:51:14.469875 6746 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "5292d011bd654339bed35e282ee662d1"
format_stamp: "Formatted at 2026-08-06 07:51:14 on dist-test-slave-g350"
I20260806 07:51:14.475131 6746 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 07:51:14.478477 6761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:14.479727 6746 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:51:14.479897 6746 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "5292d011bd654339bed35e282ee662d1"
format_stamp: "Formatted at 2026-08-06 07:51:14 on dist-test-slave-g350"
I20260806 07:51:14.480051 6746 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:14.501576 6746 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:14.505565 6746 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:14.505928 6746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:14.506906 6746 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:14.508507 6746 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:14.508579 6746 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:14.508754 6746 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:14.508811 6746 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:14.544502 6746 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:37345
I20260806 07:51:14.544585 6876 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:37345 every 8 connection(s)
I20260806 07:51:14.546113 6746 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:51:14.552707 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6746
I20260806 07:51:14.552935 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:51:14 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:14 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:14 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002674, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:14.593101 6878 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:14.593528 6878 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:14.593815 6877 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:14.593947 6877 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:14.593942 6879 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:14.594394 6879 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:14.594946 6877 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:14.595149 6878 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:14.595306 6879 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:14.596818 6351 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:37345)
I20260806 07:51:14.596923 6194 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:37345)
I20260806 07:51:14.596973 6272 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:37345)
I20260806 07:51:14.597910 6351 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:44921
I20260806 07:51:14.611708 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:14.618758 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:14.618968 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6162
I20260806 07:51:14.619748 6225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:14.728497 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6162
I20260806 07:51:14.752496 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--webserver_interface=127.20.202.254
--webserver_port=43543
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:14.983471 6732 heartbeater.cc:499] Master 127.20.202.252:41915 was elected leader, sending a full tablet report...
W20260806 07:51:14.986056 6733 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:40295 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:40295: connect: Connection refused (error 111)
W20260806 07:51:15.097916 6337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:40295: connect: Connection refused (error 111)
W20260806 07:51:15.099617 6337 consensus_peers.cc:597] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d -> Peer 9f3d2d52491f4b38b115d58397720f4c (127.20.202.254:40295): Couldn't send request to peer 9f3d2d52491f4b38b115d58397720f4c. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:40295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:51:15.107056 6888 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:15.107404 6888 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:15.107554 6888 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:15.116019 6888 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:15.116163 6888 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:15.116267 6888 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:15.116346 6888 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:15.127303 6888 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=43543
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:15.128870 6888 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:15.130414 6888 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:15.138329 6898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:15.138408 6899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:15.139794 6901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:15.140934 6888 server_base.cc:1061] running on GCE node
I20260806 07:51:15.141955 6888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:15.143822 6888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:15.145088 6888 hybrid_clock.cc:648] HybridClock initialized: now 1786002675145006 us; error 64 us; skew 500 ppm
I20260806 07:51:15.145571 6888 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:15 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:15.150125 6888 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:15.151945 6888 webserver.cc:533] Webserver started at http://127.20.202.254:43543/ using document root <none> and password file <none>
I20260806 07:51:15.152792 6888 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:15.152938 6888 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:15.158655 6888 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:51:15.161702 6908 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:15.163205 6888 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:15.163381 6888 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "9f3d2d52491f4b38b115d58397720f4c"
format_stamp: "Formatted at 2026-08-06 07:51:10 on dist-test-slave-g350"
I20260806 07:51:15.164099 6888 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:15.177558 6888 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:15.181545 6888 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:15.181880 6888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:15.204377 6888 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40295
I20260806 07:51:15.204356 6959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40295 every 8 connection(s)
I20260806 07:51:15.206116 6888 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:15.212819 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6888
I20260806 07:51:15.213146 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6453
I20260806 07:51:15.217046 6960 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:15.222935 6960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap starting.
Aug 06 07:51:15 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:51:15.244509 6582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:15.248054 6960 log.cc:826] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:15.256703 6960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:15.257220 6960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap complete.
I20260806 07:51:15.263962 6960 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:15.264663 6960 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3d2d52491f4b38b115d58397720f4c, State: Initialized, Role: FOLLOWER
I20260806 07:51:15.265368 6960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:15.266316 6966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:15.266614 6966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:15.267452 6960 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:15.278607 6977 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:15.278733 6977 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: loading cluster ID for follower catalog manager: success
I20260806 07:51:15.283728 6977 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: acquiring CA information for follower catalog manager: success
I20260806 07:51:15.286473 6977 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:15.344990 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6453
I20260806 07:51:15.365105 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41749
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--webserver_port=40911
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:15.521713 6892 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:15.522020 6892 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:15.533699 6966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:15.534006 6966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:15.600009 6877 heartbeater.cc:499] Master 127.20.202.252:41915 was elected leader, sending a full tablet report...
I20260806 07:51:15.611779 6925 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5292d011bd654339bed35e282ee662d1" instance_seqno: 1786002674536219) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:47251; Asking this server to re-register.
I20260806 07:51:15.612918 6878 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:15.614674 6925 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:37345)
W20260806 07:51:15.726603 6983 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:15.726944 6983 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:15.727088 6983 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:15.735553 6983 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:15.735812 6983 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:15.746853 6983 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40911
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:15.748510 6983 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:15.750151 6983 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:15.760291 6991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:15.760223 6992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:15.761503 6994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:15.761940 6983 server_base.cc:1061] running on GCE node
I20260806 07:51:15.762521 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:15.763494 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:15.764693 6983 hybrid_clock.cc:648] HybridClock initialized: now 1786002675764628 us; error 57 us; skew 500 ppm
I20260806 07:51:15.765012 6983 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:15 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:15.768576 6983 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:15.770313 6983 webserver.cc:533] Webserver started at http://127.20.202.193:40911/ using document root <none> and password file <none>
I20260806 07:51:15.771219 6983 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:15.771365 6983 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:15.777122 6983 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:51:15.779855 7001 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:15.781214 6983 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:15.781414 6983 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:15.782194 6983 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:15.806337 6983 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:15.810269 6983 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:15.810549 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:15.811532 6983 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:15.813124 6983 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:15.813242 6983 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:15.813371 6983 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:15.813472 6983 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:15.848765 6983 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41749
I20260806 07:51:15.848845 7116 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41749 every 8 connection(s)
I20260806 07:51:15.850476 6983 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:15.858286 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6983
Aug 06 07:51:15 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:15 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:15 dist-test-slave-g350 krb5kdc[6146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002675, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:15.892210 7118 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:15.892630 7119 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:15.892663 7118 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:15.892789 7119 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:15.893831 7118 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:15.893854 7119 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:15.894986 7117 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:15.895128 7117 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:15.895630 7117 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:15.895944 6925 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:41749)
I20260806 07:51:15.896136 6272 ts_manager.cc:194] Re-registered known tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:41749)
I20260806 07:51:15.897259 6351 ts_manager.cc:194] Re-registered known tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:41749)
I20260806 07:51:15.898134 6351 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:54445
I20260806 07:51:15.912895 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6983
W20260806 07:51:15.914168 7130 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:15.910265 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.20.202.193:41749)
0806 07:51:15.911081 (+ 816us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:15.911088 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:15.911121 (+ 33us) server_negotiation.cc:408] Connection header received
0806 07:51:15.911180 (+ 59us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:15.911186 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:15.911402 (+ 216us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:15.911569 (+ 167us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:15.912709 (+ 1140us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49932: BlockingRecv error: recv got EOF from 127.0.0.1:49932 (error 108)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":378,"threads_started":1}
W20260806 07:51:15.915336 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6983: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:41749: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:15.915508 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6983
W20260806 07:51:15.915524 7131 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:15.913714 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.20.202.193:41749)
0806 07:51:15.914340 (+ 626us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:15.914345 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:15.914371 (+ 26us) server_negotiation.cc:408] Connection header received
0806 07:51:15.914425 (+ 54us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:15.914430 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:15.914499 (+ 69us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:15.914613 (+ 114us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:15.915244 (+ 631us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49944: BlockingRecv error: recv got EOF from 127.0.0.1:49944 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":353,"threads_started":1}
I20260806 07:51:15.935053 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41749
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--webserver_port=40911
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:15.994330 6733 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:15.995368 6925 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "09e2098fd8404ca0b1ca44a8b18c02ab" instance_seqno: 1786002673893407) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:45929; Asking this server to re-register.
I20260806 07:51:15.995924 6733 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:15.996989 6925 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:42307)
I20260806 07:51:16.046602 7134 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:16.046886 7134 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:16.288101 7132 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:16.288447 7132 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:16.288600 7132 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:51:16.288691 7132 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:16.297025 7132 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:16.297292 7132 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:16.308889 7132 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40911
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:16.310695 7132 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:16.312104 7132 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:16.322322 7142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:16.322348 7143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:16.322860 7145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:16.323812 7132 server_base.cc:1061] running on GCE node
I20260806 07:51:16.324404 7132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:16.325332 7132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:16.326579 7132 hybrid_clock.cc:648] HybridClock initialized: now 1786002676326489 us; error 65 us; skew 500 ppm
I20260806 07:51:16.326870 7132 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:16 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002676, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:16.330338 7132 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:16.332062 7132 webserver.cc:533] Webserver started at http://127.20.202.193:40911/ using document root <none> and password file <none>
I20260806 07:51:16.332808 7132 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:16.332921 7132 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:16.338567 7132 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:51:16.341264 7152 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:16.342692 7132 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:16.342872 7132 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:16.343557 7132 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:16.369702 7132 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:16.374011 7132 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:16.374305 7132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:16.375272 7132 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:16.376878 7132 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:16.376979 7132 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:16.377127 7132 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:16.377204 7132 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:51:16.377568 7132 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:51:16.377668 7132 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:51:16.377759 7132 fault_injection.cc:59] injecting fault for kudu (pid 7132): FLAGS_fault_before_start (process will exit)
I20260806 07:51:16.402583 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:41749
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--webserver_port=40911
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:16.762930 7160 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:16.763293 7160 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:16.763450 7160 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:16.772142 7160 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:16.772377 7160 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:16.783856 7160 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=40911
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:16.785517 7160 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:16.786988 7160 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:16.796456 7165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:16.796499 7166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:16.797237 7168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:16.797493 7160 server_base.cc:1061] running on GCE node
I20260806 07:51:16.798116 7160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:16.799096 7160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:16.800323 7160 hybrid_clock.cc:648] HybridClock initialized: now 1786002676800271 us; error 45 us; skew 500 ppm
I20260806 07:51:16.800637 7160 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:16 dist-test-slave-g350 krb5kdc[6146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002676, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:16.803944 7160 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:16.805552 7160 webserver.cc:533] Webserver started at http://127.20.202.193:40911/ using document root <none> and password file <none>
I20260806 07:51:16.806360 7160 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:16.806494 7160 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:16.812119 7160 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:51:16.814935 7175 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:16.816354 7160 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:16.816560 7160 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:16.817329 7160 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:16.843880 7160 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:16.848093 7160 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:16.848376 7160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:16.849478 7160 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:16.851397 7160 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:16.851512 7160 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:16.851631 7160 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:16.851732 7160 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:16.887041 7160 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:41749
I20260806 07:51:16.887132 7288 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:41749 every 8 connection(s)
I20260806 07:51:16.888605 7160 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:16.889008 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7160
I20260806 07:51:16.889411 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7160
W20260806 07:51:16.923743 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7160: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:41749: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:16.923996 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7160
I20260806 07:51:16.942996 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6601
W20260806 07:51:16.943624 7294 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:16.892264 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:54243 (local address 127.20.202.254:40295)
0806 07:51:16.930653 (+ 38389us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:16.930661 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:16.930685 (+ 24us) server_negotiation.cc:408] Connection header received
0806 07:51:16.930755 (+ 70us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:16.930764 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:16.930857 (+ 93us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:16.930975 (+ 118us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:16.942697 (+ 11722us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:54243: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":38229,"thread_start_us":38402,"threads_started":1}
W20260806 07:51:16.943624 7293 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:16.891982 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:45221 (local address 127.20.202.252:41915)
0806 07:51:16.931511 (+ 39529us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:16.931519 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:16.931550 (+ 31us) server_negotiation.cc:408] Connection header received
0806 07:51:16.931614 (+ 64us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:16.931624 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:16.931709 (+ 85us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:16.931816 (+ 107us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:16.942706 (+ 10890us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:45221: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":39386,"thread_start_us":39275,"threads_started":1}
W20260806 07:51:16.944274 7300 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:16.921034 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:43907 (local address 127.20.202.253:37375)
0806 07:51:16.931761 (+ 10727us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:16.931766 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:16.942659 (+ 10893us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:43907: BlockingRecv error: recv got EOF from 127.20.202.193:43907 (error 108)
Metrics: {"server-negotiator.queue_time_us":10627,"thread_start_us":10515,"threads_started":1}
W20260806 07:51:16.945848 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6601: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:42307: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:16.946013 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6601
I20260806 07:51:16.965559 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6746
W20260806 07:51:16.968518 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6746: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:37345: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:16.968752 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6746
I20260806 07:51:16.983596 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6888
W20260806 07:51:16.985616 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6888: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:40295: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:16.985769 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6888
W20260806 07:51:16.985778 7294 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:16.984389 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:33340 (local address 127.20.202.254:40295)
0806 07:51:16.984564 (+ 175us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:16.984569 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:16.984586 (+ 17us) server_negotiation.cc:408] Connection header received
0806 07:51:16.984722 (+ 136us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:16.984727 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:16.984792 (+ 65us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:16.984879 (+ 87us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:16.985477 (+ 598us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33340: BlockingRecv error: recv got EOF from 127.0.0.1:33340 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 07:51:17.007782 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6239
I20260806 07:51:17.009023 6302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:17.118332 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6239
I20260806 07:51:17.140620 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 6318
I20260806 07:51:17.141815 6381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:17.260955 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 6318
2026-08-06T07:51:17Z chronyd exiting
Aug 06 07:51:17 dist-test-slave-g350 krb5kdc[7308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:51:17 dist-test-slave-g350 krb5kdc[7308](info): set up 2 sockets
Aug 06 07:51:17 dist-test-slave-g350 krb5kdc[7308](info): commencing operation
krb5kdc: starting...
W20260806 07:51:19.353477 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.055s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:51:19 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:51:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:19.613175 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:19.962549 7324 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:19.962930 7324 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:19.963086 7324 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:19.972052 7324 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:19.972200 7324 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:19.972318 7324 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:19.972411 7324 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:19.983551 7324 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:19.985226 7324 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:19.986776 7324 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:19.995036 7330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:19.995153 7329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:19.995749 7332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:19.996955 7324 server_base.cc:1061] running on GCE node
I20260806 07:51:19.997804 7324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:19.999295 7324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:20.000507 7324 hybrid_clock.cc:648] HybridClock initialized: now 1786002680000468 us; error 32 us; skew 500 ppm
I20260806 07:51:20.000876 7324 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:19 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:20.004642 7324 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:20.006603 7324 webserver.cc:533] Webserver started at http://127.20.202.254:37503/ using document root <none> and password file <none>
I20260806 07:51:20.007534 7324 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:20.007668 7324 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:20.013760 7324 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:51:20.016904 7340 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:20.018499 7324 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:20.018690 7324 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "9f3d2d52491f4b38b115d58397720f4c"
format_stamp: "Formatted at 2026-08-06 07:51:10 on dist-test-slave-g350"
I20260806 07:51:20.019462 7324 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:20.050215 7324 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:20.058022 7324 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:20.058358 7324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:20.080734 7324 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40295
I20260806 07:51:20.080742 7392 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40295 every 8 connection(s)
I20260806 07:51:20.082657 7324 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:20.088166 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7324
I20260806 07:51:20.088380 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:51:20.090719 7393 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:20.095963 7393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:20.114940 7393 log.cc:826] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:20.125546 7393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:20.126148 7393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap complete.
I20260806 07:51:20.133240 7393 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:20.134037 7393 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3d2d52491f4b38b115d58397720f4c, State: Initialized, Role: FOLLOWER
I20260806 07:51:20.134811 7393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:20.135764 7398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:20.136078 7398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:20.136721 7393 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:20.158607 7409 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:20.161321 7409 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: loading cluster ID for follower catalog manager: success
I20260806 07:51:20.165925 7409 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: acquiring CA information for follower catalog manager: success
I20260806 07:51:20.168977 7409 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:51:20 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:20.186484 7324 master_runner.cc:187] Error getting master registration for 127.20.202.253:37375: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37375: connect: Connection refused (error 111),
I20260806 07:51:20.192374 7324 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:20.209393 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37375
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:20.556780 7415 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:20.557166 7415 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:20.557337 7415 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:20.566126 7415 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:20.566233 7415 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:20.566355 7415 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:20.566430 7415 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:20.579254 7415 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:20.580957 7415 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:20.582455 7415 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:20.591172 7420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:20.591277 7421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:20.591950 7423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:20.593060 7415 server_base.cc:1061] running on GCE node
I20260806 07:51:20.593906 7415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:20.595304 7415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:20.596572 7415 hybrid_clock.cc:648] HybridClock initialized: now 1786002680596534 us; error 51 us; skew 500 ppm
I20260806 07:51:20.596956 7415 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:20 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002680, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:20.600557 7415 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:51:20.602227 7415 webserver.cc:533] Webserver started at http://127.20.202.253:34677/ using document root <none> and password file <none>
I20260806 07:51:20.602947 7415 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:20.603114 7415 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:20.608757 7415 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 07:51:20.611865 7430 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:20.613348 7415 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:20.613551 7415 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "21536eb957e04c83accf33b3f72ceae3"
format_stamp: "Formatted at 2026-08-06 07:51:11 on dist-test-slave-g350"
I20260806 07:51:20.614436 7415 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:20.641331 7415 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:20.645812 7415 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:20.646140 7415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:20.668203 7415 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:37375
I20260806 07:51:20.668231 7481 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:37375 every 8 connection(s)
I20260806 07:51:20.669936 7415 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:51:20.675110 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7415
I20260806 07:51:20.675344 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:51:20.678056 7482 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:20.683342 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:20.701749 7482 log.cc:826] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:20.711833 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:20.712371 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: Bootstrap complete.
I20260806 07:51:20.719658 7482 raft_consensus.cc:359] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:20.720428 7482 raft_consensus.cc:740] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21536eb957e04c83accf33b3f72ceae3, State: Initialized, Role: FOLLOWER
I20260806 07:51:20.721249 7482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:20.722203 7488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:20.722498 7488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:20.723385 7482 sys_catalog.cc:565] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:20.742067 7499 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:20.742190 7499 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: loading cluster ID for follower catalog manager: success
I20260806 07:51:20.746939 7499 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: acquiring CA information for follower catalog manager: success
I20260806 07:51:20.749536 7499 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:20.771553 7415 master_runner.cc:187] Error getting master registration for 127.20.202.252:41915: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41915: connect: Connection refused (error 111),
I20260806 07:51:20.775434 7415 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:20.800855 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41915
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:21.179445 7506 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:21.179862 7506 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:21.180033 7506 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:21.188589 7506 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:21.188738 7506 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:21.188865 7506 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:21.188961 7506 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:21.200266 7506 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:21.202059 7506 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:21.203454 7506 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:21.211799 7511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:21.211742 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:21.213256 7514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:21.213706 7506 server_base.cc:1061] running on GCE node
I20260806 07:51:21.214483 7506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:21.215956 7506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:21.217211 7506 hybrid_clock.cc:648] HybridClock initialized: now 1786002681217158 us; error 61 us; skew 500 ppm
I20260806 07:51:21.217633 7506 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:21 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002681, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:21.221314 7506 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:51:21.223057 7506 webserver.cc:533] Webserver started at http://127.20.202.252:33507/ using document root <none> and password file <none>
I20260806 07:51:21.223738 7506 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:21.223865 7506 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:21.229722 7506 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:51:21.232440 7521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:21.233850 7506 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:21.234058 7506 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "e3d38b17f71a4cf58eb0d96ebc57007d"
format_stamp: "Formatted at 2026-08-06 07:51:12 on dist-test-slave-g350"
I20260806 07:51:21.234781 7506 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:21.269764 7506 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:21.274317 7506 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:21.274608 7506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:21.295804 7506 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:41915
I20260806 07:51:21.295840 7574 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:41915 every 8 connection(s)
I20260806 07:51:21.297418 7506 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:51:21.300910 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7506
I20260806 07:51:21.301111 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:51:21.306273 7575 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:21.311450 7575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Bootstrap starting.
Aug 06 07:51:21 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:21 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:21.334223 7575 log.cc:826] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:21.344185 7575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:21.344712 7575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Bootstrap complete.
Aug 06 07:51:21 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:21.351737 7575 raft_consensus.cc:359] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:21.352466 7575 raft_consensus.cc:740] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d38b17f71a4cf58eb0d96ebc57007d, State: Initialized, Role: FOLLOWER
I20260806 07:51:21.353241 7575 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:21.354138 7582 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:21.354476 7582 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:21.355219 7575 sys_catalog.cc:565] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:21.365947 7593 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:21.366137 7593 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: loading cluster ID for follower catalog manager: success
I20260806 07:51:21.370054 7593 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: acquiring CA information for follower catalog manager: success
I20260806 07:51:21.372525 7593 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:51:21 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002681, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:21.404707 7506 master_runner.cc:264] No leader master found from master e3d38b17f71a4cf58eb0d96ebc57007d
I20260806 07:51:21.404870 7506 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:21.508530 7596 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:51:21.508846 7596 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:21.511513 7596 leader_election.cc:290] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 21536eb957e04c83accf33b3f72ceae3 (127.20.202.253:37375), e3d38b17f71a4cf58eb0d96ebc57007d (127.20.202.252:41915)
I20260806 07:51:21.520650 7549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3d2d52491f4b38b115d58397720f4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" is_pre_election: true
I20260806 07:51:21.521126 7457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3d2d52491f4b38b115d58397720f4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "21536eb957e04c83accf33b3f72ceae3" is_pre_election: true
I20260806 07:51:21.521356 7549 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3d2d52491f4b38b115d58397720f4c in term 1.
I20260806 07:51:21.521693 7457 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3d2d52491f4b38b115d58397720f4c in term 1.
I20260806 07:51:21.522068 7344 leader_election.cc:304] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3d2d52491f4b38b115d58397720f4c, e3d38b17f71a4cf58eb0d96ebc57007d; no voters:
I20260806 07:51:21.522478 7596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:51:21.522671 7596 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:51:21.522795 7596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:21.527182 7596 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:21.527930 7596 leader_election.cc:290] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [CANDIDATE]: Term 2 election: Requested vote from peers 21536eb957e04c83accf33b3f72ceae3 (127.20.202.253:37375), e3d38b17f71a4cf58eb0d96ebc57007d (127.20.202.252:41915)
I20260806 07:51:21.528386 7457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3d2d52491f4b38b115d58397720f4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "21536eb957e04c83accf33b3f72ceae3"
I20260806 07:51:21.528630 7457 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:21.528816 7549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3d2d52491f4b38b115d58397720f4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d"
I20260806 07:51:21.529032 7549 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:21.534430 7549 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3d2d52491f4b38b115d58397720f4c in term 2.
I20260806 07:51:21.534421 7457 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3d2d52491f4b38b115d58397720f4c in term 2.
I20260806 07:51:21.534893 7342 leader_election.cc:304] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21536eb957e04c83accf33b3f72ceae3, 9f3d2d52491f4b38b115d58397720f4c; no voters:
I20260806 07:51:21.535225 7596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:51:21.535797 7596 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 LEADER]: Becoming Leader. State: Replica: 9f3d2d52491f4b38b115d58397720f4c, State: Running, Role: LEADER
I20260806 07:51:21.536325 7596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:21.539096 7599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f3d2d52491f4b38b115d58397720f4c. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:21.539489 7599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:21.540243 7601 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:21.542359 7601 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:21.543174 7601 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:21.543282 7601 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:21.544432 7601 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:21.545360 7601 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Loaded TSK: 0
I20260806 07:51:21.546355 7601 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:21.674396 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:22.018582 7606 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:22.018865 7606 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:22.019053 7606 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:22.027866 7606 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:22.028091 7606 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:22.038975 7606 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:22.040800 7606 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:22.042188 7606 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:22.052199 7614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:22.052232 7611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:22.052405 7612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:22.053468 7606 server_base.cc:1061] running on GCE node
I20260806 07:51:22.054086 7606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:22.055186 7606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:22.056416 7606 hybrid_clock.cc:648] HybridClock initialized: now 1786002682056380 us; error 44 us; skew 500 ppm
I20260806 07:51:22.056761 7606 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:22.060485 7606 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:22.062178 7606 webserver.cc:533] Webserver started at http://127.20.202.193:34117/ using document root <none> and password file <none>
I20260806 07:51:22.062961 7606 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:22.063104 7606 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:22.068534 7606 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 07:51:22.071362 7621 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:22.072808 7606 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:51:22.072995 7606 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:22.073753 7606 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:22.079015 7457 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 2 FOLLOWER]: Refusing update from remote peer 9f3d2d52491f4b38b115d58397720f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:51:22.080338 7622 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:51:22.083889 7629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f3d2d52491f4b38b115d58397720f4c. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:22.084174 7629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:22.094466 7622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:22.094831 7622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:22.095777 7549 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Refusing update from remote peer 9f3d2d52491f4b38b115d58397720f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:51:22.097039 7606 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:22.097147 7629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:22.097433 7629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:22.102478 7634 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:51:22.103605 7606 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:22.103989 7606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:22.109053 7606 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:22.111061 7606 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:22.111214 7606 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:22.111348 7606 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:22.111433 7606 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:22.111642 7642 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:22.111950 7642 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:22.112162 7638 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f3d2d52491f4b38b115d58397720f4c. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:22.112476 7638 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:22.147310 7606 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38491
I20260806 07:51:22.147357 7749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38491 every 8 connection(s)
I20260806 07:51:22.148895 7606 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:22.150307 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7606
I20260806 07:51:22.150550 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:22.194360 7751 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:22.194725 7752 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:22.194810 7751 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:22.194917 7752 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:22.195768 7752 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:22.195818 7751 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:22.197991 7750 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:22.198153 7750 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:22.198267 7447 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:38491)
I20260806 07:51:22.198262 7357 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:38491)
I20260806 07:51:22.198621 7750 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:22.200249 7539 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:38491)
I20260806 07:51:22.200666 7357 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:47297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:22.277417 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:22.600611 7634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9f3d2d52491f4b38b115d58397720f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:22.600968 7634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:22.626080 7764 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:22.626366 7764 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:22.626463 7764 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:22.634783 7764 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:22.634974 7764 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:51:22.645774 7764 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:22.647404 7764 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:22.648800 7764 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:22.658797 7770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:22.658829 7769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:22.659977 7772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:22.660530 7764 server_base.cc:1061] running on GCE node
I20260806 07:51:22.661051 7764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:22.662066 7764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:22.663301 7764 hybrid_clock.cc:648] HybridClock initialized: now 1786002682663257 us; error 55 us; skew 500 ppm
I20260806 07:51:22.663647 7764 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:22.667274 7764 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:51:22.669031 7764 webserver.cc:533] Webserver started at http://127.20.202.194:43961/ using document root <none> and password file <none>
I20260806 07:51:22.669873 7764 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:22.670022 7764 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:22.675815 7764 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:51:22.678723 7779 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:22.680235 7764 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:22.680411 7764 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "09e2098fd8404ca0b1ca44a8b18c02ab"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:22.681479 7764 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:22.706571 7764 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:22.710400 7764 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:22.710713 7764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:22.711714 7764 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:22.713263 7764 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:22.713402 7764 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:22.713521 7764 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:22.713632 7764 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:22.749020 7764 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:35405
I20260806 07:51:22.749071 7894 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:35405 every 8 connection(s)
I20260806 07:51:22.750661 7764 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:51:22.755314 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7764
I20260806 07:51:22.755551 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:22 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002682, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:22.809839 7895 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:22.810345 7895 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:22.811509 7895 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:22.812556 7896 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:22.812714 7896 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:22.813251 7896 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:22.813650 7539 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:35405)
I20260806 07:51:22.813737 7897 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:22.813858 7897 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:22.814356 7897 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:22.814353 7357 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:35405)
I20260806 07:51:22.815333 7357 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:49067
I20260806 07:51:22.816056 7447 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:35405)
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:22.897437 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:23.203465 7751 heartbeater.cc:499] Master 127.20.202.254:40295 was elected leader, sending a full tablet report...
W20260806 07:51:23.236192 7909 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:23.236485 7909 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:23.236584 7909 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:23.245005 7909 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:23.245184 7909 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:51:23.255637 7909 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:23.257263 7909 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:23.258623 7909 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:23.268421 7915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:23.268455 7916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:23.269305 7918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:23.269758 7909 server_base.cc:1061] running on GCE node
I20260806 07:51:23.270321 7909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:23.271240 7909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:23.272442 7909 hybrid_clock.cc:648] HybridClock initialized: now 1786002683272395 us; error 43 us; skew 500 ppm
I20260806 07:51:23.272708 7909 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:23 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002683, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:23.276378 7909 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:51:23.278132 7909 webserver.cc:533] Webserver started at http://127.20.202.195:33589/ using document root <none> and password file <none>
I20260806 07:51:23.278879 7909 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:23.279027 7909 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:23.284605 7909 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:51:23.287415 7925 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:23.288739 7909 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:51:23.288913 7909 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "5292d011bd654339bed35e282ee662d1"
format_stamp: "Formatted at 2026-08-06 07:51:14 on dist-test-slave-g350"
I20260806 07:51:23.289601 7909 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:23.307680 7909 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:23.311679 7909 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:23.311972 7909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:23.312932 7909 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:23.314626 7909 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:23.314714 7909 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:23.314898 7909 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:23.314989 7909 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:23.350497 7909 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:34637
I20260806 07:51:23.350569 8039 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:34637 every 8 connection(s)
I20260806 07:51:23.352022 7909 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:51:23.361383 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 7909
I20260806 07:51:23.361698 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:51:23 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002683, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:23 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002683, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:23 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002683, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:23.409448 8042 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:23.409510 8041 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:23.409926 8041 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:23.409926 8042 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:23.411125 8041 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:23.411201 8042 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:23.412537 8040 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:23.412669 8040 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:23.413090 8040 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:23.413252 7447 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:34637)
I20260806 07:51:23.413938 7357 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:34637)
I20260806 07:51:23.414244 7539 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:34637)
I20260806 07:51:23.415066 7357 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:46743
I20260806 07:51:23.427074 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:23.434962 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:23.435145 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7324
I20260806 07:51:23.436062 7388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:23.550544 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7324
W20260806 07:51:23.574542 7432 connection.cc:573] server connection from 127.0.0.1:49510 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:51:23.575526 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--webserver_interface=127.20.202.254
--webserver_port=37503
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:23.817687 7896 heartbeater.cc:499] Master 127.20.202.254:40295 was elected leader, sending a full tablet report...
W20260806 07:51:23.819572 7896 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:40295 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:40295: connect: Connection refused (error 111)
W20260806 07:51:23.934598 8051 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:23.934979 8051 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:23.935130 8051 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:23.946717 8051 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:23.946849 8051 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:23.946950 8051 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:23.947037 8051 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:23.958091 8051 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=37503
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:23.959878 8051 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:23.961347 8051 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:23.969547 8057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:23.969730 8058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:23.970427 8060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:23.971238 8051 server_base.cc:1061] running on GCE node
I20260806 07:51:23.972019 8051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:23.973445 8051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:23.974668 8051 hybrid_clock.cc:648] HybridClock initialized: now 1786002683974617 us; error 71 us; skew 500 ppm
I20260806 07:51:23.975049 8051 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:23 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002683, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:23.979148 8051 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:23.981002 8051 webserver.cc:533] Webserver started at http://127.20.202.254:37503/ using document root <none> and password file <none>
I20260806 07:51:23.981829 8051 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:23.981974 8051 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:23.987782 8051 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:51:23.990782 8067 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:23.992223 8051 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:23.992412 8051 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "9f3d2d52491f4b38b115d58397720f4c"
format_stamp: "Formatted at 2026-08-06 07:51:10 on dist-test-slave-g350"
I20260806 07:51:23.993202 8051 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:24.024552 8051 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:24.028949 8051 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:24.029249 8051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:24.050985 8051 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40295
I20260806 07:51:24.051007 8118 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40295 every 8 connection(s)
I20260806 07:51:24.052749 8051 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:24.060536 8119 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:24.062685 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8051
I20260806 07:51:24.062995 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7606
I20260806 07:51:24.064603 8119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap starting.
Aug 06 07:51:24 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:51:24.084295 7745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:24.086490 8119 log.cc:826] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:24.095814 8119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:24.096369 8119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: Bootstrap complete.
I20260806 07:51:24.103443 8119 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:24.104656 8119 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3d2d52491f4b38b115d58397720f4c, State: Initialized, Role: FOLLOWER
I20260806 07:51:24.105517 8119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:24.106463 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:24.106768 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:24.107430 8119 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:24.118579 8138 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:24.118788 8138 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: loading cluster ID for follower catalog manager: success
I20260806 07:51:24.123708 8138 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: acquiring CA information for follower catalog manager: success
I20260806 07:51:24.127002 8138 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:24.162941 8051 master_runner.cc:264] No leader master found from master 9f3d2d52491f4b38b115d58397720f4c
I20260806 07:51:24.163084 8051 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:24.187094 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7606
I20260806 07:51:24.206195 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38491
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--webserver_port=34117
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:24.417353 8041 heartbeater.cc:499] Master 127.20.202.254:40295 was elected leader, sending a full tablet report...
I20260806 07:51:24.428829 8084 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5292d011bd654339bed35e282ee662d1" instance_seqno: 1786002683342173) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:59187; Asking this server to re-register.
I20260806 07:51:24.429769 8041 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:24.431529 8084 ts_manager.cc:194] Registered new tserver with Master: 5292d011bd654339bed35e282ee662d1 (127.20.202.195:34637)
W20260806 07:51:24.560663 8144 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:24.561013 8144 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:24.561169 8144 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:24.569561 8144 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:24.569833 8144 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:24.580480 8144 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=34117
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:24.582289 8144 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:24.584035 8144 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:24.594455 8150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:24.595048 8151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:24.597611 8144 server_base.cc:1061] running on GCE node
W20260806 07:51:24.598547 8153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:24.599174 8144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:24.600054 8144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:24.601253 8144 hybrid_clock.cc:648] HybridClock initialized: now 1786002684601205 us; error 36 us; skew 500 ppm
I20260806 07:51:24.601560 8144 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:24 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002684, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:24.605284 8144 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:24.606913 8144 webserver.cc:533] Webserver started at http://127.20.202.193:34117/ using document root <none> and password file <none>
I20260806 07:51:24.607676 8144 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:24.607771 8144 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:24.612949 8144 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 07:51:24.615567 8160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:24.616951 8144 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:24.617129 8144 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:24.617892 8144 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:24.654196 8144 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:24.658170 8144 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:24.658454 8144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:24.659471 8144 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:24.661244 8144 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:24.661368 8144 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:24.661489 8144 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:24.661579 8144 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:24.681941 8227 raft_consensus.cc:493] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9f3d2d52491f4b38b115d58397720f4c)
I20260806 07:51:24.682183 8227 raft_consensus.cc:515] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:24.684386 8227 leader_election.cc:290] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9f3d2d52491f4b38b115d58397720f4c (127.20.202.254:40295), 21536eb957e04c83accf33b3f72ceae3 (127.20.202.253:37375)
I20260806 07:51:24.685000 7457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "21536eb957e04c83accf33b3f72ceae3" is_pre_election: true
I20260806 07:51:24.700718 8094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9f3d2d52491f4b38b115d58397720f4c" is_pre_election: true
I20260806 07:51:24.701429 8094 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d38b17f71a4cf58eb0d96ebc57007d in term 2.
I20260806 07:51:24.702206 7524 leader_election.cc:304] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f3d2d52491f4b38b115d58397720f4c, e3d38b17f71a4cf58eb0d96ebc57007d; no voters: 21536eb957e04c83accf33b3f72ceae3
I20260806 07:51:24.702767 8227 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:51:24.702920 8227 raft_consensus.cc:493] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9f3d2d52491f4b38b115d58397720f4c)
I20260806 07:51:24.703035 8227 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:24.705130 8227 raft_consensus.cc:515] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:24.706355 8227 leader_election.cc:290] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 3 election: Requested vote from peers 9f3d2d52491f4b38b115d58397720f4c (127.20.202.254:40295), 21536eb957e04c83accf33b3f72ceae3 (127.20.202.253:37375)
I20260806 07:51:24.706316 8094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9f3d2d52491f4b38b115d58397720f4c"
I20260806 07:51:24.706431 7457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "21536eb957e04c83accf33b3f72ceae3"
I20260806 07:51:24.706532 8094 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:24.708269 8144 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38491
I20260806 07:51:24.708287 8275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38491 every 8 connection(s)
I20260806 07:51:24.710271 8144 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:24.712489 8094 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d38b17f71a4cf58eb0d96ebc57007d in term 3.
I20260806 07:51:24.713201 7524 leader_election.cc:304] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f3d2d52491f4b38b115d58397720f4c, e3d38b17f71a4cf58eb0d96ebc57007d; no voters: 21536eb957e04c83accf33b3f72ceae3
I20260806 07:51:24.714279 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8144
I20260806 07:51:24.715569 8227 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 3 FOLLOWER]: Leader election won for term 3
Aug 06 07:51:24 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002684, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:24.741017 8227 raft_consensus.cc:697] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [term 3 LEADER]: Becoming Leader. State: Replica: e3d38b17f71a4cf58eb0d96ebc57007d, State: Running, Role: LEADER
I20260806 07:51:24.741694 8227 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
Aug 06 07:51:24 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002684, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:24 dist-test-slave-g350 krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002684, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:24.758169 8277 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:24.758769 8277 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:24.760185 8277 heartbeater.cc:507] Master 127.20.202.254:40295 requested a full tablet report, sending...
I20260806 07:51:24.762835 8084 ts_manager.cc:194] Registered new tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:38491)
I20260806 07:51:24.765726 8281 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:24.766085 8281 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:24.766916 8289 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:24.769870 8289 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:24.771142 8289 catalog_manager.cc:1295] Loaded cluster ID: 0f196eca5f34462fbbaec12a291f4dc8
I20260806 07:51:24.771236 8289 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:24.772918 8289 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:24.772998 8276 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41915
I20260806 07:51:24.773166 8276 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:24.773737 8276 heartbeater.cc:507] Master 127.20.202.252:41915 requested a full tablet report, sending...
I20260806 07:51:24.774121 8289 catalog_manager.cc:6081] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d: Loaded TSK: 0
I20260806 07:51:24.775071 7539 ts_manager.cc:194] Re-registered known tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:38491)
I20260806 07:51:24.775314 8289 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:51:24.777107 8278 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37375
I20260806 07:51:24.777266 8278 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:24.777814 8278 heartbeater.cc:507] Master 127.20.202.253:37375 requested a full tablet report, sending...
I20260806 07:51:24.779016 7447 ts_manager.cc:194] Re-registered known tserver with Master: fa85193cecc843e683caf97121f8b5d0 (127.20.202.193:38491)
I20260806 07:51:24.785805 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8144
W20260806 07:51:24.787417 8291 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:24.783620 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.20.202.193:38491)
0806 07:51:24.784297 (+ 677us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:24.784303 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:24.784332 (+ 29us) server_negotiation.cc:408] Connection header received
0806 07:51:24.784383 (+ 51us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:24.784389 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:24.784597 (+ 208us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:24.784749 (+ 152us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:24.785703 (+ 954us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35938: BlockingRecv error: recv got EOF from 127.0.0.1:35938 (error 108)
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":379,"threads_started":1}
W20260806 07:51:24.789131 8291 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:24.786526 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.20.202.193:38491)
0806 07:51:24.788073 (+ 1547us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:24.788081 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:24.788106 (+ 25us) server_negotiation.cc:408] Connection header received
0806 07:51:24.788171 (+ 65us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:24.788179 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:24.788265 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:24.788393 (+ 128us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:24.788967 (+ 574us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35942: BlockingRecv error: recv got EOF from 127.0.0.1:35942 (error 108)
Metrics: {"server-negotiator.queue_time_us":1413,"thread_start_us":2228,"threads_started":1}
W20260806 07:51:24.789148 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8144: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:38491: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:24.789292 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8144
I20260806 07:51:24.808797 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38491
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--webserver_port=34117
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:24.827253 7896 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40295
I20260806 07:51:24.827500 7896 heartbeater.cc:499] Master 127.20.202.254:40295 was elected leader, sending a full tablet report...
I20260806 07:51:24.828547 8084 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "09e2098fd8404ca0b1ca44a8b18c02ab" instance_seqno: 1786002682741128) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:59185; Asking this server to re-register.
I20260806 07:51:24.829257 7896 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:24.830487 8084 ts_manager.cc:194] Registered new tserver with Master: 09e2098fd8404ca0b1ca44a8b18c02ab (127.20.202.194:35405)
I20260806 07:51:25.052382 8295 raft_consensus.cc:493] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9f3d2d52491f4b38b115d58397720f4c)
I20260806 07:51:25.052636 8295 raft_consensus.cc:515] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } }
I20260806 07:51:25.054883 8295 leader_election.cc:290] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9f3d2d52491f4b38b115d58397720f4c (127.20.202.254:40295), e3d38b17f71a4cf58eb0d96ebc57007d (127.20.202.252:41915)
I20260806 07:51:25.075052 8094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21536eb957e04c83accf33b3f72ceae3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9f3d2d52491f4b38b115d58397720f4c" is_pre_election: true
I20260806 07:51:25.075539 8094 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 21536eb957e04c83accf33b3f72ceae3 in current term 3: Already voted for candidate e3d38b17f71a4cf58eb0d96ebc57007d in this term.
I20260806 07:51:25.077139 7549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21536eb957e04c83accf33b3f72ceae3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" is_pre_election: true
I20260806 07:51:25.077847 7434 leader_election.cc:304] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21536eb957e04c83accf33b3f72ceae3; no voters: 9f3d2d52491f4b38b115d58397720f4c, e3d38b17f71a4cf58eb0d96ebc57007d
I20260806 07:51:25.078321 8295 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:25.080485 8295 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260806 07:51:25.160989 8293 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:25.161305 8293 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:25.161463 8293 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:51:25.161553 8293 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:25.169749 8293 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:25.170007 8293 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:25.180351 8293 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=34117
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:25.181988 8293 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:25.183452 8293 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:25.193316 8303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:25.193352 8302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:25.194257 8305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:25.194545 8293 server_base.cc:1061] running on GCE node
I20260806 07:51:25.195156 8293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:25.196009 8293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:25.197242 8293 hybrid_clock.cc:648] HybridClock initialized: now 1786002685197177 us; error 33 us; skew 500 ppm
I20260806 07:51:25.197618 8293 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:25 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002685, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:25.201299 8293 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:25.202950 8293 webserver.cc:533] Webserver started at http://127.20.202.193:34117/ using document root <none> and password file <none>
I20260806 07:51:25.203637 8293 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:25.203718 8293 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:25.209173 8293 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:51:25.211977 8312 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:25.213411 8293 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:51:25.213570 8293 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:25.214313 8293 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:25.228319 8293 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:25.235129 8293 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:25.235409 8293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:25.236387 8293 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:25.237985 8293 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:25.238063 8293 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:25.238234 8293 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:25.238284 8293 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:51:25.238698 8293 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:51:25.238758 8293 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:51:25.238862 8293 fault_injection.cc:59] injecting fault for kudu (pid 8293): FLAGS_fault_before_start (process will exit)
I20260806 07:51:25.249835 7457 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [term 3 FOLLOWER]: Refusing update from remote peer e3d38b17f71a4cf58eb0d96ebc57007d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:51:25.250679 8281 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:51:25.252691 8295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:25.253005 8295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:25.254738 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:25.254976 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:25.255828 8295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:25.256002 8094 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [term 3 FOLLOWER]: Refusing update from remote peer e3d38b17f71a4cf58eb0d96ebc57007d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:51:25.256048 8295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21536eb957e04c83accf33b3f72ceae3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:25.256789 8281 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:51:25.257551 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38491
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--webserver_port=34117
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:25.260458 8326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:25.260802 8326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:25.261237 8323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d38b17f71a4cf58eb0d96ebc57007d. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:25.261444 8323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3d2d52491f4b38b115d58397720f4c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:51:25.608279 8327 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:25.608659 8327 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:25.608860 8327 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:25.617477 8327 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:25.617745 8327 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:25.628942 8327 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=34117
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40295,127.20.202.253:37375,127.20.202.252:41915
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:25.630630 8327 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:25.631989 8327 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:25.642206 8333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:25.642169 8332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:25.642575 8335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:25.642901 8327 server_base.cc:1061] running on GCE node
I20260806 07:51:25.643512 8327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:25.644445 8327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:25.645697 8327 hybrid_clock.cc:648] HybridClock initialized: now 1786002685645638 us; error 59 us; skew 500 ppm
I20260806 07:51:25.646006 8327 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:25 dist-test-slave-g350 krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002685, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:25.649518 8327 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:25.651069 8327 webserver.cc:533] Webserver started at http://127.20.202.193:34117/ using document root <none> and password file <none>
I20260806 07:51:25.651778 8327 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:25.651873 8327 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:25.657414 8327 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 07:51:25.660210 8342 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:25.661681 8327 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:25.661860 8327 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "fa85193cecc843e683caf97121f8b5d0"
format_stamp: "Formatted at 2026-08-06 07:51:13 on dist-test-slave-g350"
I20260806 07:51:25.662609 8327 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:25.692554 8327 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:25.696563 8327 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:25.696856 8327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:25.698000 8327 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:25.699692 8327 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:25.699781 8327 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:25.699941 8327 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:25.700050 8327 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:25.735570 8327 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38491
I20260806 07:51:25.735636 8455 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38491 every 8 connection(s)
I20260806 07:51:25.737210 8327 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:25.740837 8281 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3d2d52491f4b38b115d58397720f4c" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40295 } } peers { permanent_uuid: "21536eb957e04c83accf33b3f72ceae3" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37375 } } peers { permanent_uuid: "e3d38b17f71a4cf58eb0d96ebc57007d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41915 } } }
I20260806 07:51:25.741185 8281 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d38b17f71a4cf58eb0d96ebc57007d [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:25.744820 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8327
I20260806 07:51:25.745153 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8327
W20260806 07:51:25.790956 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8327: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:38491: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:25.791145 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8327
I20260806 07:51:25.810564 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7764
W20260806 07:51:25.811355 8464 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:25.742033 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:37983 (local address 127.20.202.252:41915)
0806 07:51:25.786960 (+ 44927us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:25.786969 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:25.786998 (+ 29us) server_negotiation.cc:408] Connection header received
0806 07:51:25.787064 (+ 66us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:25.787072 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:25.787156 (+ 84us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:25.787273 (+ 117us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:25.810295 (+ 23022us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:37983: BlockingRecv error: recv got EOF from 127.20.202.193:37983 (error 108)
Metrics: {"server-negotiator.queue_time_us":44806,"thread_start_us":44579,"threads_started":1}
W20260806 07:51:25.811355 8462 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:25.741241 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:41675 (local address 127.20.202.253:37375)
0806 07:51:25.785513 (+ 44272us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:25.785520 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:25.785537 (+ 17us) server_negotiation.cc:408] Connection header received
0806 07:51:25.789174 (+ 3637us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:25.789190 (+ 16us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:25.789286 (+ 96us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:25.789421 (+ 135us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:25.793097 (+ 3676us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:25.794692 (+ 1595us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:25.810317 (+ 15625us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:41675: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":44156,"thread_start_us":44003,"threads_started":1}
W20260806 07:51:25.811390 8461 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:25.740891 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:44155 (local address 127.20.202.254:40295)
0806 07:51:25.784822 (+ 43931us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:25.784828 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:25.784843 (+ 15us) server_negotiation.cc:408] Connection header received
0806 07:51:25.784895 (+ 52us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:25.784902 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:25.785001 (+ 99us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:25.789671 (+ 4670us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:25.810317 (+ 20646us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:44155: BlockingRecv error: recv got EOF from 127.20.202.193:44155 (error 108)
Metrics: {"server-negotiator.queue_time_us":43811,"thread_start_us":43701,"threads_started":1}
W20260806 07:51:25.813582 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7764: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:35405: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:25.813792 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7764
I20260806 07:51:25.831861 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7909
W20260806 07:51:25.834775 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7909: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:34637: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:25.834932 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7909
I20260806 07:51:25.852491 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8051
W20260806 07:51:25.854949 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8051: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:40295: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:51:25.855054 8461 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:25.853560 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.20.202.254:40295)
0806 07:51:25.853775 (+ 215us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:25.853781 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:25.853802 (+ 21us) server_negotiation.cc:408] Connection header received
0806 07:51:25.853987 (+ 185us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:25.853992 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:25.854058 (+ 66us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:25.854171 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:25.854866 (+ 695us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44606: BlockingRecv error: recv got EOF from 127.0.0.1:44606 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 07:51:25.855154 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8051
I20260806 07:51:25.877825 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7415
I20260806 07:51:25.879105 7477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:25.997535 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7415
I20260806 07:51:26.020838 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 7506
I20260806 07:51:26.022279 7570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:26.133927 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 7506
2026-08-06T07:51:26Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/7 (17940 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/8
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:51:26 dist-test-slave-g350 krb5kdc[8475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:51:26 dist-test-slave-g350 krb5kdc[8475](info): set up 2 sockets
Aug 06 07:51:26 dist-test-slave-g350 krb5kdc[8475](info): commencing operation
krb5kdc: starting...
W20260806 07:51:28.265969 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.083s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:51:28 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:51:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:28Z Disabled control of system clock
2026-08-06T07:51:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:28Z Disabled control of system clock
2026-08-06T07:51:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:28Z Disabled control of system clock
2026-08-06T07:51:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:28Z Disabled control of system clock
2026-08-06T07:51:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:28.701258 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:29.052583 8501 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:29.052887 8501 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:29.053085 8501 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:29.061458 8501 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:29.061563 8501 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:29.061678 8501 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:29.061761 8501 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:29.072786 8501 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:29.074515 8501 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:29.076001 8501 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:29.084918 8507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:29.084885 8506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:29.085942 8509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:29.086483 8501 server_base.cc:1061] running on GCE node
I20260806 07:51:29.087281 8501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:29.088871 8501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:29.090174 8501 hybrid_clock.cc:648] HybridClock initialized: now 1786002689090035 us; error 108 us; skew 500 ppm
I20260806 07:51:29.090550 8501 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:29 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:29.093732 8501 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:29.095288 8501 webserver.cc:533] Webserver started at http://127.20.202.254:45781/ using document root <none> and password file <none>
I20260806 07:51:29.096055 8501 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:29.096200 8501 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:29.096575 8501 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:29.099045 8501 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "32d383905896405fa3f06e82957385f5"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:29.099875 8501 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "32d383905896405fa3f06e82957385f5"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:29.105203 8501 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:51:29.108610 8517 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:29.109963 8501 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:51:29.110152 8501 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "32d383905896405fa3f06e82957385f5"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:29.110365 8501 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:29.131731 8501 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:29.135808 8501 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:29.136222 8501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:29.159308 8501 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:44593
I20260806 07:51:29.159365 8568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:44593 every 8 connection(s)
I20260806 07:51:29.161094 8501 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:29.165692 8569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:51:29.167376 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8501
I20260806 07:51:29.167629 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:51:29.174717 8569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:29 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
I20260806 07:51:29.199816 8569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:51:29.201254 8518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111)
W20260806 07:51:29.203217 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40875: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
I20260806 07:51:29.252624 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } attempt: 1
W20260806 07:51:29.254817 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40875: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:29.282209 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40875
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:29.323107 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } attempt: 2
W20260806 07:51:29.325336 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40875: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111)
I20260806 07:51:29.416616 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } attempt: 3
W20260806 07:51:29.418756 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40875: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111)
I20260806 07:51:29.562076 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } attempt: 4
W20260806 07:51:29.564335 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:40875: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111)
W20260806 07:51:29.622646 8579 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:29.623003 8579 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:29.623171 8579 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:29.631839 8579 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:29.631948 8579 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:29.632009 8579 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:29.632090 8579 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:29.643769 8579 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:29.645457 8579 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:29.646926 8579 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:29.656046 8585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:29.656090 8586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:29.657153 8588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:29.657756 8579 server_base.cc:1061] running on GCE node
I20260806 07:51:29.658489 8579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:29.660149 8579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:29.661386 8579 hybrid_clock.cc:648] HybridClock initialized: now 1786002689661224 us; error 130 us; skew 500 ppm
I20260806 07:51:29.661818 8579 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:29 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:29.664999 8579 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:51:29.666675 8579 webserver.cc:533] Webserver started at http://127.20.202.253:37193/ using document root <none> and password file <none>
I20260806 07:51:29.667413 8579 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:29.667572 8579 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:29.667944 8579 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:29.670467 8579 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "098bfc418a6a4186b8b72bd231320921"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:29.671335 8579 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "098bfc418a6a4186b8b72bd231320921"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:29.676551 8579 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:51:29.679949 8595 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:29.681246 8579 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:51:29.681432 8579 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "098bfc418a6a4186b8b72bd231320921"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:29.681667 8579 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:29.712997 8579 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:29.717039 8579 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:29.717346 8579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:29.738169 8579 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:40875
I20260806 07:51:29.738193 8646 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:40875 every 8 connection(s)
I20260806 07:51:29.739696 8579 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:51:29.743914 8647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:51:29.749572 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8579
I20260806 07:51:29.749821 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:51:29.749989 8647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:29 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:29.773334 8647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:51:29 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:29.790946 8647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:51:29.792300 8597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
W20260806 07:51:29.794008 8647 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:51:29.843343 8647 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 1
W20260806 07:51:29.845464 8647 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:29.863603 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:29.866012 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35153
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 07:51:29 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:29.877734 8569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:51:29.879797 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:29.913698 8647 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 2
W20260806 07:51:29.916193 8647 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:29.931166 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 1
W20260806 07:51:29.933451 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:30.001708 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 2
W20260806 07:51:30.004068 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:30.007464 8647 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 3
W20260806 07:51:30.009680 8647 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:30.110327 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 3
W20260806 07:51:30.112615 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:30.152944 8647 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 4
W20260806 07:51:30.155227 8647 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
W20260806 07:51:30.220324 8657 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:30.220633 8657 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:30.220715 8657 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:30.228817 8657 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:30.228914 8657 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:30.228969 8657 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:30.229009 8657 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:30.239866 8657 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:30.241387 8657 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:30.242762 8657 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:30.250912 8664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:30.251046 8663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:30.251780 8666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:30.252292 8657 server_base.cc:1061] running on GCE node
I20260806 07:51:30.253019 8657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:30.254796 8657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:30.256042 8657 hybrid_clock.cc:648] HybridClock initialized: now 1786002690255899 us; error 108 us; skew 500 ppm
I20260806 07:51:30.256448 8657 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002690, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:30.259873 8657 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:51:30.261806 8657 webserver.cc:533] Webserver started at http://127.20.202.252:42219/ using document root <none> and password file <none>
I20260806 07:51:30.262583 8657 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:30.262737 8657 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:30.263108 8657 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:30.265691 8657 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "be138f97799a481bb04a739648d74c76"
format_stamp: "Formatted at 2026-08-06 07:51:30 on dist-test-slave-g350"
I20260806 07:51:30.266557 8657 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "be138f97799a481bb04a739648d74c76"
format_stamp: "Formatted at 2026-08-06 07:51:30 on dist-test-slave-g350"
I20260806 07:51:30.272341 8657 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260806 07:51:30.276087 8673 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:30.277565 8657 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:51:30.277830 8657 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "be138f97799a481bb04a739648d74c76"
format_stamp: "Formatted at 2026-08-06 07:51:30 on dist-test-slave-g350"
I20260806 07:51:30.278055 8657 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:30.289878 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 4
W20260806 07:51:30.291906 8569 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111)
I20260806 07:51:30.305328 8657 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:30.312829 8657 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:30.313144 8657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:30.335364 8657 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:35153
I20260806 07:51:30.335407 8724 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:35153 every 8 connection(s)
I20260806 07:51:30.336932 8657 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:51:30.341266 8725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:51:30.345245 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8657
I20260806 07:51:30.345489 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:51:30.348237 8725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002690, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:30.371171 8725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002690, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:30.387214 8725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002690, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:30.412844 8725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Bootstrap starting.
I20260806 07:51:30.416638 8725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Neither blocks nor log segments found. Creating new log.
I20260806 07:51:30.417672 8725 log.cc:826] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:30.420769 8725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: No bootstrap required, opened a new log
I20260806 07:51:30.426317 8725 raft_consensus.cc:359] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.426573 8725 raft_consensus.cc:385] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:51:30.426697 8725 raft_consensus.cc:740] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be138f97799a481bb04a739648d74c76, State: Initialized, Role: FOLLOWER
I20260806 07:51:30.427225 8725 consensus_queue.cc:260] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.428000 8735 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.428284 8735 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.429018 8725 sys_catalog.cc:565] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:51:30.439006 8746 catalog_manager.cc:1594] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:51:30.439214 8746 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:51:30.436281 8657 master_runner.cc:187] Error getting master registration for 127.20.202.254:44593: OK, instance_id { permanent_uuid: "32d383905896405fa3f06e82957385f5" instance_seqno: 1786002689152933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:51:30.441648 8657 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:30.454474 8647 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 5
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:30.474351 8647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Bootstrap starting.
I20260806 07:51:30.478772 8647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Neither blocks nor log segments found. Creating new log.
I20260806 07:51:30.480192 8647 log.cc:826] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:30.483635 8647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: No bootstrap required, opened a new log
I20260806 07:51:30.491490 8647 raft_consensus.cc:359] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.491833 8647 raft_consensus.cc:385] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:51:30.491967 8647 raft_consensus.cc:740] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098bfc418a6a4186b8b72bd231320921, State: Initialized, Role: FOLLOWER
I20260806 07:51:30.492684 8647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.493670 8749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.493956 8749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.494982 8647 sys_catalog.cc:565] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:51:30.506225 8760 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:51:30.506488 8760 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:51:30.503203 8579 master_runner.cc:187] Error getting master registration for 127.20.202.254:44593: OK, instance_id { permanent_uuid: "32d383905896405fa3f06e82957385f5" instance_seqno: 1786002689152933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:51:30.512017 8579 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:51:30.565681 8679 tablet.cc:2406] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:51:30.569166 8569 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } attempt: 5
Aug 06 07:51:30 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002689, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:30.587631 8569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap starting.
I20260806 07:51:30.592154 8569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Neither blocks nor log segments found. Creating new log.
I20260806 07:51:30.593303 8569 log.cc:826] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:30.596146 8569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: No bootstrap required, opened a new log
I20260806 07:51:30.601641 8569 raft_consensus.cc:359] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.601927 8569 raft_consensus.cc:385] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:51:30.602087 8569 raft_consensus.cc:740] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d383905896405fa3f06e82957385f5, State: Initialized, Role: FOLLOWER
I20260806 07:51:30.602769 8569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.603746 8763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.604122 8763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.604862 8569 sys_catalog.cc:565] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:51:30.621147 8774 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:51:30.621367 8774 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:51:30.626495 8501 master_runner.cc:264] No leader master found from master 32d383905896405fa3f06e82957385f5
I20260806 07:51:30.626642 8501 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:51:30.639526 8523 tablet.cc:2406] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:51:30.720178 8601 tablet.cc:2406] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:51:30.730741 8763 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:51:30.731035 8763 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.734220 8622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921" is_pre_election: true
I20260806 07:51:30.734730 8622 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 0.
I20260806 07:51:30.735566 8518 leader_election.cc:304] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098bfc418a6a4186b8b72bd231320921, 32d383905896405fa3f06e82957385f5; no voters:
I20260806 07:51:30.736327 8763 leader_election.cc:290] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875), be138f97799a481bb04a739648d74c76 (127.20.202.252:35153)
I20260806 07:51:30.736538 8763 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:51:30.736672 8763 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:51:30.736765 8763 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:51:30.738634 8700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be138f97799a481bb04a739648d74c76" is_pre_election: true
I20260806 07:51:30.739320 8700 raft_consensus.cc:2468] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 0.
I20260806 07:51:30.742530 8763 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.750972 8700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be138f97799a481bb04a739648d74c76"
I20260806 07:51:30.751188 8700 raft_consensus.cc:3060] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:51:30.755041 8700 raft_consensus.cc:2468] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 1.
I20260806 07:51:30.755748 8519 leader_election.cc:304] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76; no voters:
I20260806 07:51:30.756546 8763 leader_election.cc:290] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 1 election: Requested vote from peers 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875), be138f97799a481bb04a739648d74c76 (127.20.202.252:35153)
I20260806 07:51:30.756958 8763 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:51:30.757393 8763 raft_consensus.cc:697] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 LEADER]: Becoming Leader. State: Replica: 32d383905896405fa3f06e82957385f5, State: Running, Role: LEADER
I20260806 07:51:30.758149 8763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:30.773408 8622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921"
I20260806 07:51:30.773710 8622 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:51:30.777495 8622 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 1.
I20260806 07:51:30.785043 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d383905896405fa3f06e82957385f5. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.785624 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:30.792765 8778 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:30.795463 8778 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:30.816632 8700 raft_consensus.cc:1275] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Refusing update from remote peer 32d383905896405fa3f06e82957385f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:51:30.818061 8776 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:51:30.825480 8622 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 1 FOLLOWER]: Refusing update from remote peer 32d383905896405fa3f06e82957385f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:51:30.826550 8776 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:51:30.864463 8749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d383905896405fa3f06e82957385f5. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.864754 8749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.872339 8779 mvcc.cc:204] Tried to move back new op lower bound from 7315467021571686400 to 7315467021360345088. Current Snapshot: MvccSnapshot[applied={T|T < 7315467021571686400}]
I20260806 07:51:30.874047 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.874262 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:30.875034 8749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.875236 8749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.876204 8778 catalog_manager.cc:1383] Generated new cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:30.876304 8778 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:30.875991 8781 mvcc.cc:204] Tried to move back new op lower bound from 7315467021571686400 to 7315467021360345088. Current Snapshot: MvccSnapshot[applied={T|T < 7315467021571686400 or (T in {7315467021571686400})}]
I20260806 07:51:30.893528 8735 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d383905896405fa3f06e82957385f5. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.893836 8735 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.898717 8763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.898957 8763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:30.902472 8780 mvcc.cc:204] Tried to move back new op lower bound from 7315467021571686400 to 7315467021360345088. Current Snapshot: MvccSnapshot[applied={T|T < 7315467021571686400}]
I20260806 07:51:30.913769 8735 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:30.914023 8735 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:30.944150 8778 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:51:30.946034 8778 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:30.970388 8778 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Generated new TSK 0
I20260806 07:51:30.972085 8778 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:31.137368 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:31.441274 8746 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:31.441375 8746 catalog_manager.cc:1588] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: loading cluster ID for follower catalog manager: success
I20260806 07:51:31.444131 8746 catalog_manager.cc:1610] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: acquiring CA information for follower catalog manager: success
I20260806 07:51:31.446040 8746 catalog_manager.cc:1638] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:51:31.490212 8791 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:31.490562 8791 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:31.490717 8791 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:31.499087 8791 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:31.499328 8791 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:31.508410 8760 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:31.508505 8760 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: loading cluster ID for follower catalog manager: success
I20260806 07:51:31.510110 8791 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:31.511286 8760 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: acquiring CA information for follower catalog manager: success
I20260806 07:51:31.511791 8791 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:31.513230 8760 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:31.513319 8791 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:31.523007 8796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:31.523056 8798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:31.523500 8800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:31.524026 8791 server_base.cc:1061] running on GCE node
I20260806 07:51:31.524577 8791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:31.525801 8791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:31.527033 8791 hybrid_clock.cc:648] HybridClock initialized: now 1786002691526904 us; error 100 us; skew 500 ppm
I20260806 07:51:31.527364 8791 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:31 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002691, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:31.530830 8791 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:31.532472 8791 webserver.cc:533] Webserver started at http://127.20.202.193:43123/ using document root <none> and password file <none>
I20260806 07:51:31.533149 8791 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:31.533244 8791 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:31.533550 8791 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:31.536124 8791 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:31.536866 8791 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:31.542297 8791 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 07:51:31.545821 8807 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:31.547217 8791 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:51:31.547416 8791 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:31.547624 8791 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:31.561215 8791 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:31.565775 8791 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:31.566097 8791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:31.567090 8791 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:31.568709 8791 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:31.568825 8791 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:31.568974 8791 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:31.569047 8791 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:31.605166 8791 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43645
I20260806 07:51:31.605225 8923 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43645 every 8 connection(s)
I20260806 07:51:31.606725 8791 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:31.617012 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8791
I20260806 07:51:31.617251 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:51:31 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002691, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:31 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002691, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:31 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002691, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:31.649178 8925 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:31.649690 8925 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:31.650812 8925 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:31.651209 8926 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:31.651358 8926 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:31.651916 8926 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:31.653671 8534 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:43645)
I20260806 07:51:31.653826 8612 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:43645)
I20260806 07:51:31.654352 8924 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:31.654467 8924 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:31.654932 8924 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:31.655479 8534 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:42123
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
I20260806 07:51:31.656859 8690 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:43645)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:31.741477 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:32.105530 8938 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:32.105897 8938 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:32.106036 8938 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:32.114444 8938 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:32.114619 8938 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:51:32.125257 8938 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:32.126837 8938 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:32.128170 8938 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:32.138453 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:32.138516 8945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:32.139029 8947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:32.139699 8938 server_base.cc:1061] running on GCE node
I20260806 07:51:32.140259 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:32.141353 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:32.142616 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786002692142456 us; error 118 us; skew 500 ppm
I20260806 07:51:32.143020 8938 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:32.146471 8938 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:51:32.147979 8938 webserver.cc:533] Webserver started at http://127.20.202.194:43005/ using document root <none> and password file <none>
I20260806 07:51:32.148813 8938 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:32.148916 8938 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:32.149204 8938 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:32.151796 8938 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "434bea22f620483fa1b685110d89b09e"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:32.152875 8938 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "434bea22f620483fa1b685110d89b09e"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:32.158329 8938 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:51:32.161759 8954 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:32.163162 8938 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:51:32.163354 8938 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "434bea22f620483fa1b685110d89b09e"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:32.163563 8938 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:32.189906 8938 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:32.194104 8938 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:32.194418 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:32.195417 8938 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:32.197145 8938 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:32.197254 8938 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:32.197403 8938 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:32.197474 8938 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:32.232877 8938 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:33277
I20260806 07:51:32.232930 9067 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:33277 every 8 connection(s)
I20260806 07:51:32.234406 8938 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:51:32.238415 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 8938
I20260806 07:51:32.238660 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
I20260806 07:51:32.295291 9070 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:32.295713 9068 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:32.295769 9070 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:32.295881 9068 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:32.297024 9070 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:32.297135 9068 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.195@KRBTEST.COM" created.
I20260806 07:51:32.299152 8612 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:33277)
I20260806 07:51:32.299147 8690 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:33277)
I20260806 07:51:32.300025 9069 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:32.300194 9069 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:32.300632 9069 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:32.301638 8534 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:33277)
I20260806 07:51:32.302453 8534 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:40753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:32.381769 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:32.658187 8925 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
W20260806 07:51:32.718529 9082 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:32.718875 9082 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:32.719038 9082 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:32.727356 9082 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:32.727602 9082 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:51:32.738906 9082 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:32.740868 9082 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:32.742612 9082 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:32.754631 9088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:32.754902 9089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:32.755007 9091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:32.756064 9082 server_base.cc:1061] running on GCE node
I20260806 07:51:32.756642 9082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:32.757845 9082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:32.759099 9082 hybrid_clock.cc:648] HybridClock initialized: now 1786002692758921 us; error 144 us; skew 500 ppm
I20260806 07:51:32.759438 9082 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:32.762820 9082 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:51:32.764353 9082 webserver.cc:533] Webserver started at http://127.20.202.195:45017/ using document root <none> and password file <none>
I20260806 07:51:32.765136 9082 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:32.765231 9082 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:32.765573 9082 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:51:32.768047 9082 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "8e1688fda71e477ea8ac1ffb9049c792"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:32.768823 9082 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "8e1688fda71e477ea8ac1ffb9049c792"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:32.774343 9082 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 07:51:32.777889 9098 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:32.779340 9082 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:51:32.779577 9082 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "8e1688fda71e477ea8ac1ffb9049c792"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:32.779799 9082 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:32.797238 9082 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:32.801172 9082 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:32.801460 9082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:32.802431 9082 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:32.804025 9082 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:32.804107 9082 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:32.804275 9082 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:32.804324 9082 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:32.839843 9082 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:42833
I20260806 07:51:32.839898 9211 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:42833 every 8 connection(s)
I20260806 07:51:32.841285 9082 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:51:32.846542 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9082
I20260806 07:51:32.846786 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:32 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002692, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:32.885398 9214 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:32.885414 9212 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:32.885950 9212 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:32.885946 9214 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:32.886861 9213 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:32.887042 9213 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:32.887233 9214 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:32.887662 9213 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:32.888324 9212 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:32.889490 8611 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:42833)
I20260806 07:51:32.889679 8690 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:42833)
I20260806 07:51:32.890152 8534 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:42833)
I20260806 07:51:32.891218 8534 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:59817
I20260806 07:51:32.893461 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:32.901129 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:32.901330 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8501
I20260806 07:51:32.902199 8564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:33.014356 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8501
W20260806 07:51:33.038534 8598 connection.cc:573] server connection from 127.0.0.1:57270 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:51:33.041803 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--webserver_interface=127.20.202.254
--webserver_port=45781
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:33.304606 9069 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
W20260806 07:51:33.306286 9069 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:44593 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:44593: connect: Connection refused (error 111)
W20260806 07:51:33.388836 9223 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:33.389217 9223 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:33.389366 9223 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:33.397811 9223 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:33.397941 9223 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:33.398066 9223 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:33.398146 9223 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:33.408915 9223 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=45781
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:33.410642 9223 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:33.412101 9223 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:33.421006 9229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:33.421249 9232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:33.421734 9230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:33.422592 9223 server_base.cc:1061] running on GCE node
I20260806 07:51:33.423338 9223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:33.424938 9223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:33.426177 9223 hybrid_clock.cc:648] HybridClock initialized: now 1786002693426023 us; error 122 us; skew 500 ppm
I20260806 07:51:33.426534 9223 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:33 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002693, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:33.429948 9223 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:33.431563 9223 webserver.cc:533] Webserver started at http://127.20.202.254:45781/ using document root <none> and password file <none>
I20260806 07:51:33.432307 9223 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:33.432461 9223 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:33.439575 9223 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 07:51:33.443806 9239 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:33.445770 9223 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 07:51:33.445958 9223 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "32d383905896405fa3f06e82957385f5"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:33.447054 9223 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:33.469673 9223 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:33.474453 9223 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:33.474714 9223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:33.495934 9223 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:44593
I20260806 07:51:33.495954 9290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:44593 every 8 connection(s)
I20260806 07:51:33.497522 9223 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:33.497977 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9223
I20260806 07:51:33.498301 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8791
I20260806 07:51:33.506331 9291 sys_catalog.cc:263] Verifying existing consensus state
Aug 06 07:51:33 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:51:33.509761 9291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap starting.
I20260806 07:51:33.516785 8918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:33.531432 9291 log.cc:826] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:33.542441 9291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:33.543052 9291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap complete.
I20260806 07:51:33.553090 9291 raft_consensus.cc:359] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:33.554060 9291 raft_consensus.cc:740] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d383905896405fa3f06e82957385f5, State: Initialized, Role: FOLLOWER
I20260806 07:51:33.554898 9291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:33.555749 9297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:33.556032 9297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:33.556978 9291 sys_catalog.cc:565] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:33.567611 9308 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:33.567765 9308 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: loading cluster ID for follower catalog manager: success
I20260806 07:51:33.572674 9308 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: acquiring CA information for follower catalog manager: success
I20260806 07:51:33.575760 9308 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:33.611723 9223 master_runner.cc:264] No leader master found from master 32d383905896405fa3f06e82957385f5
I20260806 07:51:33.611861 9223 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:33.633229 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8791
I20260806 07:51:33.652913 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43645
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--webserver_port=43123
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:33.894025 9213 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
I20260806 07:51:33.905683 9256 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8e1688fda71e477ea8ac1ffb9049c792" instance_seqno: 1786002692831389) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:44049; Asking this server to re-register.
I20260806 07:51:33.906670 9213 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:33.908509 9256 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:42833)
W20260806 07:51:34.013691 9314 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:34.013986 9314 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:34.014079 9314 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:34.022652 9314 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:34.022828 9314 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:34.033555 9314 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43123
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:34.035136 9314 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:34.036652 9314 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:34.047072 9320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:34.047111 9321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:34.050901 9323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:34.051646 9314 server_base.cc:1061] running on GCE node
I20260806 07:51:34.052222 9314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:34.053292 9314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:34.056593 9314 hybrid_clock.cc:648] HybridClock initialized: now 1786002694056420 us; error 136 us; skew 500 ppm
I20260806 07:51:34.056970 9314 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:34 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:34.060813 9314 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:34.062867 9314 webserver.cc:533] Webserver started at http://127.20.202.193:43123/ using document root <none> and password file <none>
I20260806 07:51:34.063663 9314 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:34.063809 9314 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:34.069443 9314 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:51:34.072216 9330 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:34.073760 9314 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:34.073939 9314 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:34.074669 9314 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:34.082074 9335 raft_consensus.cc:493] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 32d383905896405fa3f06e82957385f5)
I20260806 07:51:34.082304 9335 raft_consensus.cc:515] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:34.083835 9335 leader_election.cc:290] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32d383905896405fa3f06e82957385f5 (127.20.202.254:44593), 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875)
I20260806 07:51:34.084589 8622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921" is_pre_election: true
I20260806 07:51:34.091037 9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "32d383905896405fa3f06e82957385f5" is_pre_election: true
I20260806 07:51:34.091516 9266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be138f97799a481bb04a739648d74c76 in term 1.
I20260806 07:51:34.092113 8676 leader_election.cc:304] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76; no voters: 098bfc418a6a4186b8b72bd231320921
I20260806 07:51:34.092514 9335 raft_consensus.cc:2804] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:51:34.092641 9335 raft_consensus.cc:493] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 32d383905896405fa3f06e82957385f5)
I20260806 07:51:34.092729 9335 raft_consensus.cc:3060] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:34.095803 9335 raft_consensus.cc:515] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:34.096509 9335 leader_election.cc:290] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 2 election: Requested vote from peers 32d383905896405fa3f06e82957385f5 (127.20.202.254:44593), 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875)
I20260806 07:51:34.097019 8622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921"
I20260806 07:51:34.097031 9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "32d383905896405fa3f06e82957385f5"
I20260806 07:51:34.097225 9266 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:34.102300 9266 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be138f97799a481bb04a739648d74c76 in term 2.
I20260806 07:51:34.102725 8676 leader_election.cc:304] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76; no voters: 098bfc418a6a4186b8b72bd231320921
I20260806 07:51:34.103044 9335 raft_consensus.cc:2804] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:51:34.103507 9335 raft_consensus.cc:697] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 LEADER]: Becoming Leader. State: Replica: be138f97799a481bb04a739648d74c76, State: Running, Role: LEADER
I20260806 07:51:34.103961 9335 consensus_queue.cc:237] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:34.105571 9337 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be138f97799a481bb04a739648d74c76. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:34.105916 9337 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:34.106681 9339 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:34.108402 9339 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:34.109254 9339 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:34.109354 9339 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:34.110601 9339 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:34.111411 9339 catalog_manager.cc:6081] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Loaded TSK: 0
I20260806 07:51:34.112270 9339 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:51:34.116070 9314 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:34.119841 9314 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:34.120117 9314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:34.121109 9314 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:34.122743 9314 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:34.122869 9314 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:34.123008 9314 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:34.123098 9314 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:34.159328 9314 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43645
I20260806 07:51:34.159416 9450 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43645 every 8 connection(s)
I20260806 07:51:34.161181 9314 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:34.161573 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9314
Aug 06 07:51:34 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:34 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:34 dist-test-slave-g350 krb5kdc[8475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:34.193408 9452 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:34.194002 9452 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:34.194996 9452 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:34.196296 9451 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:34.196430 9451 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:34.196907 9451 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:34.198226 8689 ts_manager.cc:194] Re-registered known tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:43645)
I20260806 07:51:34.200086 8689 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:47481
I20260806 07:51:34.201663 9256 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:43645)
I20260806 07:51:34.201710 9453 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:34.201869 9453 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:34.202392 9453 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:34.203703 8611 ts_manager.cc:194] Re-registered known tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:43645)
I20260806 07:51:34.217393 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9314
W20260806 07:51:34.218693 9463 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:34.214949 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:40604 (local address 127.20.202.193:43645)
0806 07:51:34.215596 (+ 647us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:34.215607 (+ 11us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:34.215670 (+ 63us) server_negotiation.cc:408] Connection header received
0806 07:51:34.215885 (+ 215us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:34.215893 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:34.216098 (+ 205us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:34.216287 (+ 189us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:34.217237 (+ 950us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40604: BlockingRecv error: recv got EOF from 127.0.0.1:40604 (error 108)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":316,"threads_started":1}
W20260806 07:51:34.219867 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9314: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:43645: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:34.220036 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9314
I20260806 07:51:34.238561 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43645
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--webserver_port=43123
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:34.314639 9069 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:34.314904 9069 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
I20260806 07:51:34.315853 9256 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "434bea22f620483fa1b685110d89b09e" instance_seqno: 1786002692224406) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:56815; Asking this server to re-register.
I20260806 07:51:34.316519 9069 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:34.317837 9256 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:33277)
I20260806 07:51:34.515504 9467 raft_consensus.cc:493] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 32d383905896405fa3f06e82957385f5)
I20260806 07:51:34.515794 9467 raft_consensus.cc:515] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:34.517163 9467 leader_election.cc:290] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32d383905896405fa3f06e82957385f5 (127.20.202.254:44593), be138f97799a481bb04a739648d74c76 (127.20.202.252:35153)
I20260806 07:51:34.518035 8699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "098bfc418a6a4186b8b72bd231320921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be138f97799a481bb04a739648d74c76" is_pre_election: true
I20260806 07:51:34.521168 9266 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Refusing update from remote peer be138f97799a481bb04a739648d74c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:51:34.522156 9337 consensus_queue.cc:1048] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:51:34.525259 9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "098bfc418a6a4186b8b72bd231320921" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "32d383905896405fa3f06e82957385f5" is_pre_election: true
I20260806 07:51:34.526230 8598 leader_election.cc:304] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098bfc418a6a4186b8b72bd231320921; no voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76
I20260806 07:51:34.526645 9467 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:51:34.526638 9469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be138f97799a481bb04a739648d74c76. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:34.526985 9469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:34.529517 9469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:34.529881 9469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:34.533895 8622 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:51:34.538146 9335 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:34.538444 9335 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:34.538445 8622 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 2 FOLLOWER]: Refusing update from remote peer be138f97799a481bb04a739648d74c76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:51:34.539049 9335 consensus_queue.cc:1048] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:51:34.542110 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:34.542248 9467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be138f97799a481bb04a739648d74c76. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:34.542554 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:34.542608 9467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:51:34.587299 9465 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:34.587603 9465 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:34.587705 9465 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:51:34.587764 9465 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:34.596097 9465 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:34.596398 9465 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:34.607259 9465 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43123
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:34.608877 9465 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:34.610402 9465 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:34.620584 9482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:34.621551 9484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:34.621552 9481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:34.622541 9465 server_base.cc:1061] running on GCE node
I20260806 07:51:34.623318 9465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:34.624408 9465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:34.625710 9465 hybrid_clock.cc:648] HybridClock initialized: now 1786002694625536 us; error 116 us; skew 500 ppm
I20260806 07:51:34.626051 9465 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:34 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002694, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:34.629369 9465 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:34.631126 9465 webserver.cc:533] Webserver started at http://127.20.202.193:43123/ using document root <none> and password file <none>
I20260806 07:51:34.631861 9465 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:34.631994 9465 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:34.637482 9465 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:34.640142 9491 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:34.641876 9465 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:34.642030 9465 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:34.642683 9465 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:34.676046 9465 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:34.679903 9465 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:34.680155 9465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:34.681102 9465 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:34.682869 9465 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:34.682955 9465 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:34.683068 9465 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:34.683120 9465 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:51:34.683569 9465 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:51:34.683634 9465 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:51:34.683713 9465 fault_injection.cc:59] injecting fault for kudu (pid 9465): FLAGS_fault_before_start (process will exit)
I20260806 07:51:34.705276 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43645
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--webserver_port=43123
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:35.045728 9500 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:35.046067 9500 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:35.046219 9500 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:35.054641 9500 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:35.054869 9500 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:35.065477 9500 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35421,127.20.202.213:44073,127.20.202.214:37425,127.20.202.215:33331,127.20.202.216:40517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43123
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:35.067152 9500 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:35.068508 9500 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:51:35.075071 9335 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:35.075366 9335 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:35.079023 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:35.079066 9510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:35.079757 9512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:35.080924 9500 server_base.cc:1061] running on GCE node
I20260806 07:51:35.081540 9500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:35.082800 9500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:35.084041 9500 hybrid_clock.cc:648] HybridClock initialized: now 1786002695083913 us; error 95 us; skew 500 ppm
I20260806 07:51:35.084395 9500 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:35 dist-test-slave-g350 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002695, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:35.088091 9500 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:35.089829 9500 webserver.cc:533] Webserver started at http://127.20.202.193:43123/ using document root <none> and password file <none>
I20260806 07:51:35.090634 9500 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:35.090775 9500 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:35.096602 9500 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 07:51:35.099617 9519 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:35.100981 9500 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:51:35.101176 9500 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:35.101984 9500 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:35.115849 9500 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:35.119829 9500 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:35.120102 9500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:35.121064 9500 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:35.122575 9500 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:35.122670 9500 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:35.122747 9500 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:35.122805 9500 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:35.157451 9500 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43645
I20260806 07:51:35.157546 9635 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43645 every 8 connection(s)
I20260806 07:51:35.159199 9500 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:35.161334 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9500
I20260806 07:51:35.161859 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9500
W20260806 07:51:35.214833 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9500: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:43645: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:35.215054 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9500
I20260806 07:51:35.233683 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8938
W20260806 07:51:35.234534 9642 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:35.163687 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:51529 (local address 127.20.202.253:40875)
0806 07:51:35.176116 (+ 12429us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:35.176124 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:35.204016 (+ 27892us) server_negotiation.cc:408] Connection header received
0806 07:51:35.204158 (+ 142us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:35.204163 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:35.204249 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:35.204357 (+ 108us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:35.207406 (+ 3049us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:35.208883 (+ 1477us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:35.213315 (+ 4432us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:35.213859 (+ 544us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:51:35.233518 (+ 19659us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:51529: BlockingRecv error: recv got EOF from 127.20.202.193:51529 (error 108)
Metrics: {"server-negotiator.queue_time_us":12239,"thread_start_us":12434,"threads_started":1}
W20260806 07:51:35.234534 9643 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:35.164028 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:33885 (local address 127.20.202.252:35153)
0806 07:51:35.202480 (+ 38452us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:35.202489 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:35.203188 (+ 699us) server_negotiation.cc:408] Connection header received
0806 07:51:35.203363 (+ 175us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:35.203371 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:35.203464 (+ 93us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:35.203601 (+ 137us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:35.212155 (+ 8554us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:35.213563 (+ 1408us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:35.233503 (+ 19940us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:33885: BlockingRecv error: recv got EOF from 127.20.202.193:33885 (error 108)
Metrics: {"server-negotiator.queue_time_us":38264,"thread_start_us":38475,"threads_started":1}
W20260806 07:51:35.235993 9644 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:35.164321 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:45391 (local address 127.20.202.254:44593)
0806 07:51:35.199801 (+ 35480us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:35.199811 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:35.199843 (+ 32us) server_negotiation.cc:408] Connection header received
0806 07:51:35.206254 (+ 6411us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:35.206261 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:35.206344 (+ 83us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:35.206459 (+ 115us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:35.208154 (+ 1695us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:35.209722 (+ 1568us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:35.212170 (+ 2448us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:35.212811 (+ 641us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:51:35.235326 (+ 22515us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:45391: BlockingRecv error: recv got EOF from 127.20.202.193:45391 (error 108)
Metrics: {"server-negotiator.queue_time_us":35308,"thread_start_us":35556,"threads_started":1}
W20260806 07:51:35.238224 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8938: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:33277: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:35.238351 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8938
I20260806 07:51:35.253211 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9082
W20260806 07:51:35.256338 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9082: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:42833: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:35.256518 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9082
I20260806 07:51:35.273802 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9223
W20260806 07:51:35.276021 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9223: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:44593: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:35.276168 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9223
W20260806 07:51:35.276221 9644 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:35.274836 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.20.202.254:44593)
0806 07:51:35.275004 (+ 168us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:35.275008 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:35.275032 (+ 24us) server_negotiation.cc:408] Connection header received
0806 07:51:35.275120 (+ 88us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:35.275125 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:35.275191 (+ 66us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:35.275274 (+ 83us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:35.275919 (+ 645us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49274: BlockingRecv error: recv got EOF from 127.0.0.1:49274 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 07:51:35.298794 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8579
I20260806 07:51:35.300066 8642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:35.422515 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8579
I20260806 07:51:35.443492 8657 master_runner.cc:187] Error getting master registration for 127.20.202.254:44593: Network error: Client connection negotiation failed: client connection to 127.20.202.254:44593: connect: Connection refused (error 111),
I20260806 07:51:35.443759 8657 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:35.445737 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 8657
I20260806 07:51:35.446887 8720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:35.568133 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 8657
2026-08-06T07:51:35Z chronyd exiting
2026-08-06T07:51:35Z chronyd exiting
2026-08-06T07:51:35Z chronyd exiting
2026-08-06T07:51:35Z chronyd exiting
2026-08-06T07:51:35Z chronyd exiting
Aug 06 07:51:35 dist-test-slave-g350 krb5kdc[9655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:51:35 dist-test-slave-g350 krb5kdc[9655](info): set up 2 sockets
Aug 06 07:51:35 dist-test-slave-g350 krb5kdc[9655](info): commencing operation
krb5kdc: starting...
W20260806 07:51:37.647753 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.000s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:51:37 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:51:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:37Z Disabled control of system clock
2026-08-06T07:51:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:37Z Disabled control of system clock
2026-08-06T07:51:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:37Z Disabled control of system clock
2026-08-06T07:51:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:37Z Disabled control of system clock
2026-08-06T07:51:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:38.048683 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:38.397055 9679 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:38.397403 9679 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:38.397578 9679 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:38.406153 9679 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:38.406261 9679 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:38.406390 9679 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:38.406466 9679 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:38.417572 9679 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:38.419248 9679 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:38.420687 9679 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:38.429071 9685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:38.429622 9684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:38.430521 9687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:38.431284 9679 server_base.cc:1061] running on GCE node
I20260806 07:51:38.432101 9679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:38.433777 9679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:38.435011 9679 hybrid_clock.cc:648] HybridClock initialized: now 1786002698434871 us; error 109 us; skew 500 ppm
I20260806 07:51:38.435382 9679 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:38 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002698, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:38.438887 9679 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:38.440588 9679 webserver.cc:533] Webserver started at http://127.20.202.254:37627/ using document root <none> and password file <none>
I20260806 07:51:38.441352 9679 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:38.441500 9679 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:38.447145 9679 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:51:38.450172 9694 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:38.451694 9679 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:51:38.451889 9679 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "32d383905896405fa3f06e82957385f5"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:38.452628 9679 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:38.471844 9679 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:38.483384 9679 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:38.483704 9679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:38.505180 9679 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:44593
I20260806 07:51:38.505198 9747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:44593 every 8 connection(s)
I20260806 07:51:38.506942 9679 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:38.514853 9748 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:38.515156 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9679
I20260806 07:51:38.515359 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:51:38.518482 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap starting.
I20260806 07:51:38.535181 9748 log.cc:826] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:38.544099 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:38.544623 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap complete.
I20260806 07:51:38.551724 9748 raft_consensus.cc:359] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:38.552464 9748 raft_consensus.cc:740] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d383905896405fa3f06e82957385f5, State: Initialized, Role: FOLLOWER
I20260806 07:51:38.553192 9748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:38.554141 9753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:38.554466 9753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:38.555405 9748 sys_catalog.cc:565] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:38.566525 9764 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:38.566650 9764 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: loading cluster ID for follower catalog manager: success
I20260806 07:51:38.571776 9764 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: acquiring CA information for follower catalog manager: success
I20260806 07:51:38.574327 9764 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Aug 06 07:51:38 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002698, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:38.595908 9679 master_runner.cc:187] Error getting master registration for 127.20.202.253:40875: Network error: Client connection negotiation failed: client connection to 127.20.202.253:40875: connect: Connection refused (error 111),
I20260806 07:51:38.601868 9679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:38.633536 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40875
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:38.982820 9771 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:38.983134 9771 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:38.983316 9771 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:38.991901 9771 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:38.991999 9771 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:38.992089 9771 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:38.992182 9771 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:39.003177 9771 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:40875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:39.004849 9771 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:39.006289 9771 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:39.014433 9777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:39.014469 9776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:39.015067 9779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:39.016103 9771 server_base.cc:1061] running on GCE node
I20260806 07:51:39.016812 9771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:39.018518 9771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:39.019739 9771 hybrid_clock.cc:648] HybridClock initialized: now 1786002699019563 us; error 127 us; skew 500 ppm
I20260806 07:51:39.020108 9771 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002699, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:39.023559 9771 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:51:39.025245 9771 webserver.cc:533] Webserver started at http://127.20.202.253:37831/ using document root <none> and password file <none>
I20260806 07:51:39.026070 9771 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:39.026206 9771 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:39.031920 9771 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 07:51:39.035178 9786 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:39.036615 9771 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:39.036796 9771 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "098bfc418a6a4186b8b72bd231320921"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:39.037516 9771 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:39.052663 9771 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:39.056740 9771 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:39.057003 9771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:39.078635 9771 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:40875
I20260806 07:51:39.078666 9838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:40875 every 8 connection(s)
I20260806 07:51:39.080287 9771 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:51:39.081202 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9771
I20260806 07:51:39.081468 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:51:39.087481 9839 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:39.090799 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:39.107496 9839 log.cc:826] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:39.118549 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:39.119204 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: Bootstrap complete.
I20260806 07:51:39.130753 9839 raft_consensus.cc:359] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:39.131611 9839 raft_consensus.cc:740] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098bfc418a6a4186b8b72bd231320921, State: Initialized, Role: FOLLOWER
I20260806 07:51:39.132350 9839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:39.133205 9844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:39.133477 9844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:39.134301 9839 sys_catalog.cc:565] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:39.160300 9855 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:39.160430 9855 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: loading cluster ID for follower catalog manager: success
I20260806 07:51:39.164105 9855 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: acquiring CA information for follower catalog manager: success
I20260806 07:51:39.166724 9855 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002699, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:39.194892 9771 master_runner.cc:187] Error getting master registration for 127.20.202.252:35153: Network error: Client connection negotiation failed: client connection to 127.20.202.252:35153: connect: Connection refused (error 111),
I20260806 07:51:39.201802 9771 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:39.220542 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35153
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:51:39.580261 9863 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:39.580559 9863 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:39.580641 9863 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:39.588914 9863 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:39.589013 9863 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:39.589069 9863 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:39.589123 9863 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:39.599668 9863 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:35153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:39.601210 9863 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:39.602532 9863 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:39.610682 9868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:39.610719 9869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:39.611483 9871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:39.611861 9863 server_base.cc:1061] running on GCE node
I20260806 07:51:39.612578 9863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:39.614130 9863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:39.615365 9863 hybrid_clock.cc:648] HybridClock initialized: now 1786002699615217 us; error 110 us; skew 500 ppm
I20260806 07:51:39.615727 9863 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002699, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:39.619190 9863 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:51:39.620699 9863 webserver.cc:533] Webserver started at http://127.20.202.252:43077/ using document root <none> and password file <none>
I20260806 07:51:39.621356 9863 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:39.621470 9863 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:39.626670 9863 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:51:39.629544 9878 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:39.630995 9863 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:51:39.631198 9863 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "be138f97799a481bb04a739648d74c76"
format_stamp: "Formatted at 2026-08-06 07:51:30 on dist-test-slave-g350"
I20260806 07:51:39.631910 9863 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:39.648170 9863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:39.652381 9863 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:39.652663 9863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:39.674746 9863 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:35153
I20260806 07:51:39.674830 9929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:35153 every 8 connection(s)
I20260806 07:51:39.676445 9863 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:51:39.680349 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9863
I20260806 07:51:39.680562 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:51:39.684412 9930 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:39.688211 9930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Bootstrap starting.
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:51:39.706008 9930 log.cc:826] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:39.717401 9930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:39.718106 9930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Bootstrap complete.
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:39.725273 9930 raft_consensus.cc:359] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:39.726367 9930 raft_consensus.cc:740] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: be138f97799a481bb04a739648d74c76, State: Initialized, Role: FOLLOWER
I20260806 07:51:39.727228 9930 consensus_queue.cc:260] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:39.728135 9936 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:39.728464 9936 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:39.729370 9930 sys_catalog.cc:565] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:39.740731 9947 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:39.740908 9947 catalog_manager.cc:1588] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: loading cluster ID for follower catalog manager: success
Aug 06 07:51:39 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002699, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:39.746414 9947 catalog_manager.cc:1610] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: acquiring CA information for follower catalog manager: success
I20260806 07:51:39.749110 9947 catalog_manager.cc:1638] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:39.775422 9863 master_runner.cc:264] No leader master found from master be138f97799a481bb04a739648d74c76
I20260806 07:51:39.775547 9863 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:39.993829 9950 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:51:39.994313 9950 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:39.997537 9950 leader_election.cc:290] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875), be138f97799a481bb04a739648d74c76 (127.20.202.252:35153)
I20260806 07:51:40.007151 9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921" is_pre_election: true
I20260806 07:51:40.007557 9905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be138f97799a481bb04a739648d74c76" is_pre_election: true
I20260806 07:51:40.007944 9813 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 2.
I20260806 07:51:40.008229 9905 raft_consensus.cc:2468] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 2.
I20260806 07:51:40.008816 9696 leader_election.cc:304] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098bfc418a6a4186b8b72bd231320921, 32d383905896405fa3f06e82957385f5; no voters:
I20260806 07:51:40.009361 9950 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:51:40.009541 9950 raft_consensus.cc:493] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:51:40.009681 9950 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:40.013398 9950 raft_consensus.cc:515] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:40.014163 9950 leader_election.cc:290] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 3 election: Requested vote from peers 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875), be138f97799a481bb04a739648d74c76 (127.20.202.252:35153)
I20260806 07:51:40.014803 9905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "be138f97799a481bb04a739648d74c76"
I20260806 07:51:40.015015 9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d383905896405fa3f06e82957385f5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921"
I20260806 07:51:40.015086 9905 raft_consensus.cc:3060] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:40.015259 9813 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:51:40.018851 9813 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 3.
I20260806 07:51:40.019322 9696 leader_election.cc:304] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098bfc418a6a4186b8b72bd231320921, 32d383905896405fa3f06e82957385f5; no voters:
I20260806 07:51:40.019660 9950 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:51:40.020176 9950 raft_consensus.cc:697] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 LEADER]: Becoming Leader. State: Replica: 32d383905896405fa3f06e82957385f5, State: Running, Role: LEADER
I20260806 07:51:40.020706 9905 raft_consensus.cc:2468] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d383905896405fa3f06e82957385f5 in term 3.
I20260806 07:51:40.020700 9950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:40.023870 9953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d383905896405fa3f06e82957385f5. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.024154 9953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:40.024916 9955 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:40.026999 9955 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:40.027788 9955 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:40.027894 9955 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:40.028995 9955 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:40.029848 9955 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Loaded TSK: 0
I20260806 07:51:40.031010 9955 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:40.175096 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:40.488090 9813 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 3 FOLLOWER]: Refusing update from remote peer 32d383905896405fa3f06e82957385f5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:51:40.489151 9950 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:51:40.493340 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d383905896405fa3f06e82957385f5. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.493687 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:40.494743 9950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.495021 9950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:40.496838 9905 raft_consensus.cc:1275] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Refusing update from remote peer 32d383905896405fa3f06e82957385f5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:51:40.497915 9950 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:51:40.513788 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.514031 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:40.521164 9970 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.521989 9967 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d383905896405fa3f06e82957385f5. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.522383 9967 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:40.522526 9970 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:51:40.532577 9960 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:40.532919 9960 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:40.533089 9960 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:40.543799 9960 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:40.544040 9960 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:40.554948 9960 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:40.556799 9960 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:40.558332 9960 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:40.568022 9975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:40.569204 9976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:40.569422 9978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:40.570581 9960 server_base.cc:1061] running on GCE node
I20260806 07:51:40.571149 9960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:40.572265 9960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:40.573518 9960 hybrid_clock.cc:648] HybridClock initialized: now 1786002700573349 us; error 116 us; skew 500 ppm
I20260806 07:51:40.573887 9960 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:40 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002700, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:40.577466 9960 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:40.579311 9960 webserver.cc:533] Webserver started at http://127.20.202.193:46323/ using document root <none> and password file <none>
I20260806 07:51:40.580134 9960 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:40.580274 9960 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:40.585866 9960 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:51:40.588541 9985 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:40.590020 9960 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:40.590204 9960 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:40.590915 9960 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:40.628361 9960 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:40.632808 9960 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:40.633085 9960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:40.634176 9960 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:40.635914 9960 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:40.636034 9960 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:40.636170 9960 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:40.636255 9960 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:40.673997 9960 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33407
I20260806 07:51:40.674031 10099 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33407 every 8 connection(s)
I20260806 07:51:40.675961 9960 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:40.682271 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 9960
I20260806 07:51:40.682509 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:51:40 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002700, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:51:40 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002700, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:40 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002700, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:51:40.731408 10100 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:40.731788 10100 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:40.732647 10100 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:40.735488 9895 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:33407)
I20260806 07:51:40.738111 10102 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:40.738276 10102 heartbeater.cc:461] Registering TS with master...
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
I20260806 07:51:40.738834 10102 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
add_principal: Principal or policy already exists while creating "kudu/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:40.740777 10101 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:40.740936 10101 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:40.740897 9803 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:33407)
I20260806 07:51:40.741338 10101 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:40.742913 9712 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:33407)
I20260806 07:51:40.744882 9712 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:60361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:40.823347 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:40.938572 9965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "32d383905896405fa3f06e82957385f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:40.938935 9965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: LEADER
W20260806 07:51:41.180490 10114 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:41.180832 10114 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:41.181011 10114 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:41.189365 10114 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:41.189647 10114 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:51:41.200372 10114 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:41.202226 10114 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:41.203612 10114 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:41.214238 10120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:41.214251 10122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:41.214651 10114 server_base.cc:1061] running on GCE node
W20260806 07:51:41.216055 10119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:41.216751 10114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:41.217916 10114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:41.219194 10114 hybrid_clock.cc:648] HybridClock initialized: now 1786002701219059 us; error 104 us; skew 500 ppm
I20260806 07:51:41.219535 10114 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:41.223397 10114 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:51:41.224933 10114 webserver.cc:533] Webserver started at http://127.20.202.194:33839/ using document root <none> and password file <none>
I20260806 07:51:41.225711 10114 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:41.225831 10114 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:41.231400 10114 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:51:41.234217 10129 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:41.235641 10114 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:41.235805 10114 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "434bea22f620483fa1b685110d89b09e"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:41.236616 10114 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:41.260238 10114 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:41.264468 10114 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:41.264775 10114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:41.265928 10114 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:41.267524 10114 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:41.267616 10114 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:41.267781 10114 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:41.267872 10114 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:41.302562 10114 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:43663
I20260806 07:51:41.302654 10243 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:43663 every 8 connection(s)
I20260806 07:51:41.304116 10114 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:51:41.310894 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 10114
I20260806 07:51:41.311124 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:41.355836 10246 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:41.356365 10246 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:41.357445 10245 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:41.357697 10245 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:41.357731 10246 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:41.358178 10244 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:41.358367 10245 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:41.358384 10244 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:41.359033 10244 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:41.360127 9712 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:43663)
I20260806 07:51:41.360230 9803 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:43663)
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:51:41.361294 9712 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:45667
I20260806 07:51:41.361819 9895 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:43663)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:51:41.443193 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:41.747730 10101 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
W20260806 07:51:41.797168 10258 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:41.797545 10258 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:41.797719 10258 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:41.806169 10258 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:41.806429 10258 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:51:41.817289 10258 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:41.818976 10258 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:41.820389 10258 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:41.830312 10264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:41.830677 10263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:41.831288 10266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:41.832187 10258 server_base.cc:1061] running on GCE node
I20260806 07:51:41.832774 10258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:41.834115 10258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:41.837450 10258 hybrid_clock.cc:648] HybridClock initialized: now 1786002701837233 us; error 185 us; skew 500 ppm
I20260806 07:51:41.837836 10258 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:41.841523 10258 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:51:41.843326 10258 webserver.cc:533] Webserver started at http://127.20.202.195:33735/ using document root <none> and password file <none>
I20260806 07:51:41.844065 10258 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:41.844200 10258 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:41.849870 10258 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:51:41.852582 10273 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:41.854096 10258 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:41.854274 10258 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "8e1688fda71e477ea8ac1ffb9049c792"
format_stamp: "Formatted at 2026-08-06 07:51:32 on dist-test-slave-g350"
I20260806 07:51:41.855151 10258 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:41.875831 10258 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:41.879637 10258 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:41.879944 10258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:41.880885 10258 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:41.882428 10258 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:41.882512 10258 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:41.882591 10258 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:41.882643 10258 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:41.917487 10258 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:35089
I20260806 07:51:41.917546 10387 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:35089 every 8 connection(s)
I20260806 07:51:41.919082 10258 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:51:41.920866 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 10258
I20260806 07:51:41.921108 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:41 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:41.955837 10391 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:41.956293 10389 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:41.956314 10391 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:41.956470 10389 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:41.956663 10390 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:41.956844 10390 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:41.957458 10391 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:41.957630 10390 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:41.957645 10389 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:41.959555 9895 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:35089)
I20260806 07:51:41.959786 9712 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:35089)
I20260806 07:51:41.959951 9802 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:35089)
I20260806 07:51:41.960939 9712 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:53415
I20260806 07:51:41.972101 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:41.979149 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:51:41.979322 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9679
I20260806 07:51:41.980163 9743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:42.099081 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9679
W20260806 07:51:42.123203 9788 connection.cc:573] server connection from 127.0.0.1:40390 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:51:42.124264 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--webserver_interface=127.20.202.254
--webserver_port=37627
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:42.363852 10245 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
W20260806 07:51:42.365527 10245 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:44593 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:44593: connect: Connection refused (error 111)
W20260806 07:51:42.477216 10400 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:42.477581 10400 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:42.477777 10400 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:42.486286 10400 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:51:42.486441 10400 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:42.486552 10400 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:51:42.486634 10400 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:51:42.497720 10400 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=37627
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:42.499449 10400 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:42.500861 10400 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:42.510331 10406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:42.510372 10407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:42.510558 10409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:42.511548 10400 server_base.cc:1061] running on GCE node
I20260806 07:51:42.512351 10400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:42.514153 10400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:42.515380 10400 hybrid_clock.cc:648] HybridClock initialized: now 1786002702515246 us; error 99 us; skew 500 ppm
I20260806 07:51:42.515805 10400 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:42 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002702, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:42.519858 10400 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:51:42.521524 10400 webserver.cc:533] Webserver started at http://127.20.202.254:37627/ using document root <none> and password file <none>
I20260806 07:51:42.522341 10400 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:42.522460 10400 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:42.528411 10400 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:51:42.531522 10416 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:42.533203 10400 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:42.533416 10400 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "32d383905896405fa3f06e82957385f5"
format_stamp: "Formatted at 2026-08-06 07:51:29 on dist-test-slave-g350"
I20260806 07:51:42.534333 10400 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:42.551854 10400 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:42.556219 10400 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:42.556535 10400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:42.577914 10400 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:44593
I20260806 07:51:42.577935 10470 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:44593 every 8 connection(s)
I20260806 07:51:42.579571 10400 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:51:42.581094 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 10400
I20260806 07:51:42.581463 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9960
I20260806 07:51:42.587528 10471 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:51:42.591197 10471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap starting.
Aug 06 07:51:42 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:51:42.600495 10095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:42.611004 10471 log.cc:826] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Log is configured to *not* fsync() on all Append() calls
I20260806 07:51:42.620772 10471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:51:42.621356 10471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: Bootstrap complete.
I20260806 07:51:42.628410 10471 raft_consensus.cc:359] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:42.629149 10471 raft_consensus.cc:740] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d383905896405fa3f06e82957385f5, State: Initialized, Role: FOLLOWER
I20260806 07:51:42.630285 10471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:42.631276 10477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:42.631597 10477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:42.632436 10471 sys_catalog.cc:565] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:51:42.644102 10488 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:42.644218 10488 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: loading cluster ID for follower catalog manager: success
I20260806 07:51:42.649106 10488 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: acquiring CA information for follower catalog manager: success
I20260806 07:51:42.652064 10488 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:51:42.703605 10400 master_runner.cc:264] No leader master found from master 32d383905896405fa3f06e82957385f5
I20260806 07:51:42.703768 10400 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:51:42.714608 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9960
I20260806 07:51:42.740756 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33407
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--webserver_port=46323
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:42.962986 10390 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
I20260806 07:51:42.974556 10433 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8e1688fda71e477ea8ac1ffb9049c792" instance_seqno: 1786002701908994) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:46767; Asking this server to re-register.
I20260806 07:51:42.975657 10390 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:42.977499 10433 ts_manager.cc:194] Registered new tserver with Master: 8e1688fda71e477ea8ac1ffb9049c792 (127.20.202.195:35089)
W20260806 07:51:43.098731 10494 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:43.099133 10494 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:43.099291 10494 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:43.107821 10494 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:43.108088 10494 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:43.119499 10494 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46323
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:43.121300 10494 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:43.122879 10494 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:43.133028 10500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:43.133636 10501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:43.133692 10503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:43.135535 10494 server_base.cc:1061] running on GCE node
I20260806 07:51:43.136217 10494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:43.137460 10494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:43.138792 10494 hybrid_clock.cc:648] HybridClock initialized: now 1786002703138601 us; error 157 us; skew 500 ppm
I20260806 07:51:43.139158 10494 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:43 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:43.143167 10494 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:43.144860 10494 webserver.cc:533] Webserver started at http://127.20.202.193:46323/ using document root <none> and password file <none>
I20260806 07:51:43.145582 10494 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:43.145761 10494 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:43.151501 10494 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 07:51:43.154309 10510 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:43.155733 10494 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:51:43.155902 10494 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:43.156600 10494 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:43.178982 10494 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:43.183130 10494 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:43.183419 10494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:43.184445 10494 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:43.186113 10494 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:43.186203 10494 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:43.186357 10494 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:43.186461 10494 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:43.223145 10494 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33407
I20260806 07:51:43.223196 10624 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33407 every 8 connection(s)
I20260806 07:51:43.224680 10494 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:43.230108 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 10494
Aug 06 07:51:43 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:51:43 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:51:43 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:43.264983 10626 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:43.265357 10626 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:43.266551 10626 heartbeater.cc:507] Master 127.20.202.254:44593 requested a full tablet report, sending...
I20260806 07:51:43.268800 10433 ts_manager.cc:194] Registered new tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:33407)
I20260806 07:51:43.274026 10627 heartbeater.cc:344] Connected to a master server at 127.20.202.253:40875
I20260806 07:51:43.274185 10627 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:43.274699 10627 heartbeater.cc:507] Master 127.20.202.253:40875 requested a full tablet report, sending...
I20260806 07:51:43.275171 10625 heartbeater.cc:344] Connected to a master server at 127.20.202.252:35153
I20260806 07:51:43.275303 10625 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:43.275769 10625 heartbeater.cc:507] Master 127.20.202.252:35153 requested a full tablet report, sending...
I20260806 07:51:43.276033 9802 ts_manager.cc:194] Re-registered known tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:33407)
I20260806 07:51:43.276975 9895 ts_manager.cc:194] Re-registered known tserver with Master: e7815039149e4993ad6f6c8ed895dc86 (127.20.202.193:33407)
I20260806 07:51:43.288457 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10494
W20260806 07:51:43.289728 10637 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:43.285636 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.20.202.193:33407)
0806 07:51:43.286907 (+ 1271us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:43.286916 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:43.286943 (+ 27us) server_negotiation.cc:408] Connection header received
0806 07:51:43.286994 (+ 51us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:43.287000 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:43.287200 (+ 200us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:43.287343 (+ 143us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:43.288361 (+ 1018us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47426: BlockingRecv error: recv got EOF from 127.0.0.1:47426 (error 108)
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":977,"threads_started":1}
W20260806 07:51:43.291146 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10494: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33407: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:43.291287 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 10494
W20260806 07:51:43.291311 10638 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:43.289378 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.20.202.193:33407)
0806 07:51:43.290108 (+ 730us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:43.290112 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:43.290129 (+ 17us) server_negotiation.cc:408] Connection header received
0806 07:51:43.290167 (+ 38us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:43.290171 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:43.290220 (+ 49us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:43.290338 (+ 118us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:43.291087 (+ 749us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47438: BlockingRecv error: recv got EOF from 127.0.0.1:47438 (error 108)
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":482,"threads_started":1}
I20260806 07:51:43.310477 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33407
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--webserver_port=46323
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:43.374405 10245 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44593
I20260806 07:51:43.374639 10245 heartbeater.cc:499] Master 127.20.202.254:44593 was elected leader, sending a full tablet report...
I20260806 07:51:43.375787 10433 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "434bea22f620483fa1b685110d89b09e" instance_seqno: 1786002701294011) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:40135; Asking this server to re-register.
I20260806 07:51:43.376551 10245 heartbeater.cc:461] Registering TS with master...
I20260806 07:51:43.377943 10433 ts_manager.cc:194] Registered new tserver with Master: 434bea22f620483fa1b685110d89b09e (127.20.202.194:43663)
I20260806 07:51:43.448450 10641 raft_consensus.cc:493] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 32d383905896405fa3f06e82957385f5)
I20260806 07:51:43.448720 10641 raft_consensus.cc:515] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:43.450676 10641 leader_election.cc:290] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 32d383905896405fa3f06e82957385f5 (127.20.202.254:44593), 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875)
I20260806 07:51:43.451471 9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921" is_pre_election: true
I20260806 07:51:43.458811 10444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "32d383905896405fa3f06e82957385f5" is_pre_election: true
I20260806 07:51:43.459470 10444 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be138f97799a481bb04a739648d74c76 in term 3.
I20260806 07:51:43.460080 9881 leader_election.cc:304] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76; no voters: 098bfc418a6a4186b8b72bd231320921
I20260806 07:51:43.460503 10641 raft_consensus.cc:2804] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 07:51:43.460661 10641 raft_consensus.cc:493] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 32d383905896405fa3f06e82957385f5)
I20260806 07:51:43.460795 10641 raft_consensus.cc:3060] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:51:43.463030 10641 raft_consensus.cc:515] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:43.463733 10641 leader_election.cc:290] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 4 election: Requested vote from peers 32d383905896405fa3f06e82957385f5 (127.20.202.254:44593), 098bfc418a6a4186b8b72bd231320921 (127.20.202.253:40875)
I20260806 07:51:43.464172 10444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "32d383905896405fa3f06e82957385f5"
I20260806 07:51:43.464438 10444 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:51:43.464529 9813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be138f97799a481bb04a739648d74c76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "098bfc418a6a4186b8b72bd231320921"
I20260806 07:51:43.468291 10444 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be138f97799a481bb04a739648d74c76 in term 4.
I20260806 07:51:43.468761 9881 leader_election.cc:304] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76; no voters: 098bfc418a6a4186b8b72bd231320921
I20260806 07:51:43.469143 10641 raft_consensus.cc:2804] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 4 FOLLOWER]: Leader election won for term 4
I20260806 07:51:43.469655 10641 raft_consensus.cc:697] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [term 4 LEADER]: Becoming Leader. State: Replica: be138f97799a481bb04a739648d74c76, State: Running, Role: LEADER
I20260806 07:51:43.470252 10641 consensus_queue.cc:237] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:43.472497 10643 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be138f97799a481bb04a739648d74c76. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:43.472852 10643 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:43.473680 10645 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:51:43.475617 10645 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:51:43.476469 10645 catalog_manager.cc:1295] Loaded cluster ID: f10db7613a0440f19c39a2469dd3111c
I20260806 07:51:43.476539 10645 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:51:43.477726 10645 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:51:43.478544 10645 catalog_manager.cc:6081] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76: Loaded TSK: 0
I20260806 07:51:43.479350 10645 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:51:43.601054 10646 raft_consensus.cc:493] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 32d383905896405fa3f06e82957385f5)
I20260806 07:51:43.601277 10646 raft_consensus.cc:515] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } }
I20260806 07:51:43.603250 10646 leader_election.cc:290] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 32d383905896405fa3f06e82957385f5 (127.20.202.254:44593), be138f97799a481bb04a739648d74c76 (127.20.202.252:35153)
I20260806 07:51:43.611984 9905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "098bfc418a6a4186b8b72bd231320921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "be138f97799a481bb04a739648d74c76" is_pre_election: true
I20260806 07:51:43.612298 10444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "098bfc418a6a4186b8b72bd231320921" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "32d383905896405fa3f06e82957385f5" is_pre_election: true
I20260806 07:51:43.612709 10444 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098bfc418a6a4186b8b72bd231320921 in current term 4: Already voted for candidate be138f97799a481bb04a739648d74c76 in this term.
I20260806 07:51:43.613448 9789 leader_election.cc:304] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098bfc418a6a4186b8b72bd231320921; no voters: 32d383905896405fa3f06e82957385f5, be138f97799a481bb04a739648d74c76
I20260806 07:51:43.613884 10646 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 3 FOLLOWER]: Advancing to term 4
I20260806 07:51:43.616132 10646 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260806 07:51:43.693337 10639 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:43.693728 10639 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:43.693884 10639 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:51:43.693979 10639 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:43.703115 10639 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:43.703378 10639 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:43.714155 10639 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46323
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:43.715832 10639 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:43.717236 10639 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:43.727201 10655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:43.727262 10657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:43.727416 10654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:43.728499 10639 server_base.cc:1061] running on GCE node
I20260806 07:51:43.729090 10639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:43.730350 10639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:43.731591 10639 hybrid_clock.cc:648] HybridClock initialized: now 1786002703731435 us; error 109 us; skew 500 ppm
I20260806 07:51:43.731911 10639 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:43 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:43.735869 10639 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:43.737846 10639 webserver.cc:533] Webserver started at http://127.20.202.193:46323/ using document root <none> and password file <none>
I20260806 07:51:43.738698 10639 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:43.738824 10639 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:43.744760 10639 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 07:51:43.747648 10664 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:43.749013 10639 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:51:43.749192 10639 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:43.749970 10639 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:43.768469 10639 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:43.772630 10639 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:43.772945 10639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:43.774138 10639 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:43.775843 10639 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:43.775951 10639 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:43.776104 10639 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:43.776186 10639 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:51:43.776592 10639 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:51:43.776681 10639 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:51:43.776765 10639 fault_injection.cc:59] injecting fault for kudu (pid 10639): FLAGS_fault_before_start (process will exit)
I20260806 07:51:43.800673 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33407
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--webserver_port=46323
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:51:43.927908 9813 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [term 4 FOLLOWER]: Refusing update from remote peer be138f97799a481bb04a739648d74c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:51:43.928795 10643 consensus_queue.cc:1048] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:51:43.930948 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be138f97799a481bb04a739648d74c76. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:43.931226 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:43.933064 10641 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:43.933370 10641 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:43.934541 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:43.934713 10444 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [term 4 FOLLOWER]: Refusing update from remote peer be138f97799a481bb04a739648d74c76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260806 07:51:43.934753 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098bfc418a6a4186b8b72bd231320921 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:43.935524 10641 consensus_queue.cc:1048] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:51:43.939301 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:43.939672 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:51:43.942836 10678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be138f97799a481bb04a739648d74c76. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:43.943177 10678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d383905896405fa3f06e82957385f5 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:51:44.153424 10672 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:51:44.153784 10672 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:51:44.153956 10672 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:51:44.162554 10672 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:51:44.162853 10672 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:51:44.174638 10672 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36205,127.20.202.213:38477,127.20.202.214:44415,127.20.202.215:43293,127.20.202.216:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46323
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44593,127.20.202.253:40875,127.20.202.252:35153
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:51:44.176301 10672 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:51:44.177789 10672 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:51:44.187522 10686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:44.187548 10687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:51:44.188395 10689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:51:44.189618 10672 server_base.cc:1061] running on GCE node
I20260806 07:51:44.190229 10672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:51:44.191387 10672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:51:44.192668 10672 hybrid_clock.cc:648] HybridClock initialized: now 1786002704192517 us; error 116 us; skew 500 ppm
I20260806 07:51:44.193022 10672 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:51:44 dist-test-slave-g350 krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002704, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:51:44.196810 10672 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:51:44.198444 10672 webserver.cc:533] Webserver started at http://127.20.202.193:46323/ using document root <none> and password file <none>
I20260806 07:51:44.199128 10672 fs_manager.cc:362] Metadata directory not provided
I20260806 07:51:44.199226 10672 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:51:44.205317 10672 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 07:51:44.209152 10696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:44.210911 10672 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:51:44.211053 10672 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e7815039149e4993ad6f6c8ed895dc86"
format_stamp: "Formatted at 2026-08-06 07:51:31 on dist-test-slave-g350"
I20260806 07:51:44.211742 10672 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:51:44.239156 10672 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:51:44.243548 10672 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:51:44.243836 10672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:51:44.244858 10672 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:51:44.246582 10672 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:51:44.246672 10672 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:44.246836 10672 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:51:44.246933 10672 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:51:44.282740 10672 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33407
I20260806 07:51:44.282825 10809 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33407 every 8 connection(s)
I20260806 07:51:44.284336 10672 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:51:44.288556 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 10672
I20260806 07:51:44.288851 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10672
Aug 06 07:51:44 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002704, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
W20260806 07:51:44.320955 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10672: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33407: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:44.321132 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 10672
Aug 06 07:51:44 dist-test-slave-g350 krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002704, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:51:44.340116 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10114
W20260806 07:51:44.340843 10815 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:44.287948 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:39455 (local address 127.20.202.254:44593)
0806 07:51:44.311470 (+ 23522us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:44.311479 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:44.314291 (+ 2812us) server_negotiation.cc:408] Connection header received
0806 07:51:44.314365 (+ 74us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:44.314370 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:44.314453 (+ 83us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:44.314561 (+ 108us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:44.316222 (+ 1661us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:44.318050 (+ 1828us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:44.339786 (+ 21736us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:39455: BlockingRecv error: recv got EOF from 127.20.202.193:39455 (error 108)
Metrics: {"server-negotiator.queue_time_us":23370,"thread_start_us":23214,"threads_started":1}
W20260806 07:51:44.340842 10816 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:44.288142 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:43595 (local address 127.20.202.252:35153)
0806 07:51:44.311834 (+ 23692us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:44.311840 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:44.311859 (+ 19us) server_negotiation.cc:408] Connection header received
0806 07:51:44.311931 (+ 72us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:44.311937 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:44.312016 (+ 79us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:44.312150 (+ 134us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:44.313569 (+ 1419us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:44.315031 (+ 1462us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:44.318655 (+ 3624us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:44.319199 (+ 544us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:51:44.339814 (+ 20615us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:43595: BlockingRecv error: recv got EOF from 127.20.202.193:43595 (error 108)
Metrics: {"server-negotiator.queue_time_us":23590,"thread_start_us":23494,"threads_started":1}
W20260806 07:51:44.340865 10814 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:51:44.287797 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:48695 (local address 127.20.202.253:40875)
0806 07:51:44.288366 (+ 569us) server_negotiation.cc:207] Beginning negotiation
0806 07:51:44.288374 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:51:44.304308 (+ 15934us) server_negotiation.cc:408] Connection header received
0806 07:51:44.306695 (+ 2387us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:51:44.306705 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:51:44.306824 (+ 119us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:51:44.306986 (+ 162us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:51:44.312201 (+ 5215us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:44.313989 (+ 1788us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:51:44.316933 (+ 2944us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:51:44.317625 (+ 692us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:51:44.339813 (+ 22188us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:48695: BlockingRecv error: recv got EOF from 127.20.202.193:48695 (error 108)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":421,"threads_started":1}
W20260806 07:51:44.343652 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10114: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:43663: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:44.343840 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 10114
I20260806 07:51:44.353754 10675 sys_catalog.cc:455] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "be138f97799a481bb04a739648d74c76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d383905896405fa3f06e82957385f5" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44593 } } peers { permanent_uuid: "098bfc418a6a4186b8b72bd231320921" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 40875 } } peers { permanent_uuid: "be138f97799a481bb04a739648d74c76" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 35153 } } }
I20260806 07:51:44.354100 10675 sys_catalog.cc:458] T 00000000000000000000000000000000 P be138f97799a481bb04a739648d74c76 [sys.catalog]: This master's current role is: LEADER
I20260806 07:51:44.359864 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10258
W20260806 07:51:44.362865 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10258: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:35089: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:44.363046 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 10258
I20260806 07:51:44.380723 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10400
W20260806 07:51:44.382966 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 10400: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:44593: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:51:44.383147 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 10400
I20260806 07:51:44.407167 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9771
I20260806 07:51:44.408417 9834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:44.521788 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9771
I20260806 07:51:44.546351 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 9863
I20260806 07:51:44.547736 9925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:51:44.658318 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 9863
2026-08-06T07:51:44Z chronyd exiting
2026-08-06T07:51:44Z chronyd exiting
2026-08-06T07:51:44Z chronyd exiting
2026-08-06T07:51:44Z chronyd exiting
2026-08-06T07:51:44Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/8 (18566 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/9
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:51:44 dist-test-slave-g350 krb5kdc[10830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:51:44 dist-test-slave-g350 krb5kdc[10830](info): set up 2 sockets
Aug 06 07:51:44 dist-test-slave-g350 krb5kdc[10830](info): commencing operation
krb5kdc: starting...
W20260806 07:51:46.814918 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.066s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:51:46 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:51:55: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:51:57 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002717, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:52:03 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002723, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:52:03 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002723, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:05.878018 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:06.239378 11129 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:06.239708 11129 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:06.239849 11129 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:06.248399 11129 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:06.248507 11129 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:06.248564 11129 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:06.248612 11129 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:06.259754 11129 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:06.261468 11129 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:06.262948 11129 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:06.272140 11135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:06.272289 11137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:06.272078 11134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:06.273272 11129 server_base.cc:1061] running on GCE node
I20260806 07:52:06.274168 11129 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:06.274361 11129 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:06.274459 11129 hybrid_clock.cc:648] HybridClock initialized: now 1786002726274419 us; error 0 us; skew 500 ppm
I20260806 07:52:06.274854 11129 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:06 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:06.278333 11129 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:52:06.280371 11129 webserver.cc:533] Webserver started at http://127.20.202.254:35483/ using document root <none> and password file <none>
I20260806 07:52:06.281142 11129 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:06.281273 11129 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:06.281735 11129 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:06.284283 11129 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "c336a69c59f14d7da636e9a1b1fff164"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:06.285131 11129 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "c336a69c59f14d7da636e9a1b1fff164"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:06.290863 11129 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 07:52:06.294320 11143 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:06.295787 11129 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:52:06.295977 11129 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c336a69c59f14d7da636e9a1b1fff164"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:06.296175 11129 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:06.317787 11129 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:06.322042 11129 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:06.322355 11129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:06.344820 11129 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40139
I20260806 07:52:06.344853 11195 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40139 every 8 connection(s)
I20260806 07:52:06.346522 11129 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:52:06.351157 11196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:52:06.354101 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11129
I20260806 07:52:06.354326 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:52:06.357842 11196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:06 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
I20260806 07:52:06.384974 11196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:52:06.386380 11145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111)
W20260806 07:52:06.388106 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38221: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:06.437510 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } attempt: 1
W20260806 07:52:06.439812 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38221: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:06.473088 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38221
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:06.508095 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } attempt: 2
W20260806 07:52:06.510179 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38221: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111)
I20260806 07:52:06.601532 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } attempt: 3
W20260806 07:52:06.603833 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38221: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111)
I20260806 07:52:06.747140 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } attempt: 4
W20260806 07:52:06.749577 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38221: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111)
W20260806 07:52:06.837497 11205 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:06.837990 11205 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:06.838152 11205 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:06.846971 11205 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:06.847088 11205 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:06.847185 11205 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:06.847263 11205 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:06.858850 11205 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:06.860646 11205 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:06.862144 11205 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:06.870664 11210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:06.871181 11211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:06.871438 11213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:06.872022 11205 server_base.cc:1061] running on GCE node
I20260806 07:52:06.872804 11205 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:06.872946 11205 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:06.873063 11205 hybrid_clock.cc:648] HybridClock initialized: now 1786002726873040 us; error 0 us; skew 500 ppm
I20260806 07:52:06.873409 11205 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:06 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:06.876813 11205 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:52:06.878929 11205 webserver.cc:533] Webserver started at http://127.20.202.253:38943/ using document root <none> and password file <none>
I20260806 07:52:06.879757 11205 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:06.879894 11205 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:06.880256 11205 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:06.882771 11205 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "28b9f7ed0d5745e19c092f948a7b7d39"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:06.883607 11205 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "28b9f7ed0d5745e19c092f948a7b7d39"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:06.889067 11205 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 07:52:06.892700 11219 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:06.894189 11205 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:06.894371 11205 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "28b9f7ed0d5745e19c092f948a7b7d39"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:06.894559 11205 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:06.927770 11205 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:06.931912 11205 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:06.932247 11205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:06.954085 11205 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:38221
I20260806 07:52:06.954119 11270 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:38221 every 8 connection(s)
I20260806 07:52:06.955729 11205 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:52:06.960084 11271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:52:06.961555 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11205
I20260806 07:52:06.961794 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:52:06.966607 11271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:06 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
I20260806 07:52:06.990366 11271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:52:06 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:07.007128 11271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:52:07.008311 11222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
W20260806 07:52:07.010110 11271 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:07.048928 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } attempt: 5
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:07.059506 11271 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 1
W20260806 07:52:07.061843 11271 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.065485 11196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:52:07.067518 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:07.086825 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38497
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:07.118786 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 1
W20260806 07:52:07.120956 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.130074 11271 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 2
W20260806 07:52:07.132236 11271 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.189217 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 2
W20260806 07:52:07.191438 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.223511 11271 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 3
W20260806 07:52:07.225835 11271 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.297677 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 3
W20260806 07:52:07.300141 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.369055 11271 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 4
W20260806 07:52:07.371378 11271 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
W20260806 07:52:07.439340 11281 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:07.439718 11281 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:07.439874 11281 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:07.448179 11281 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:07.448338 11281 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:07.448472 11281 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:07.448559 11281 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:07.459457 11281 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:07.461156 11281 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:07.462549 11281 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:07.470765 11286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:07.470816 11287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:07.471733 11289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:07.472646 11281 server_base.cc:1061] running on GCE node
I20260806 07:52:07.473414 11281 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:07.473630 11281 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:07.473743 11281 hybrid_clock.cc:648] HybridClock initialized: now 1786002727473702 us; error 0 us; skew 500 ppm
I20260806 07:52:07.474104 11281 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002727, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:07.477362 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 4
I20260806 07:52:07.477922 11281 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
W20260806 07:52:07.479525 11196 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111)
I20260806 07:52:07.480331 11281 webserver.cc:533] Webserver started at http://127.20.202.252:40473/ using document root <none> and password file <none>
I20260806 07:52:07.481088 11281 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:07.481240 11281 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:07.481643 11281 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:07.484650 11281 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "ca65838c4315482791c4a8a1c9bee3aa"
format_stamp: "Formatted at 2026-08-06 07:52:07 on dist-test-slave-g350"
I20260806 07:52:07.485527 11281 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "ca65838c4315482791c4a8a1c9bee3aa"
format_stamp: "Formatted at 2026-08-06 07:52:07 on dist-test-slave-g350"
I20260806 07:52:07.491122 11281 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 07:52:07.494563 11295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:07.495965 11281 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:52:07.496141 11281 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "ca65838c4315482791c4a8a1c9bee3aa"
format_stamp: "Formatted at 2026-08-06 07:52:07 on dist-test-slave-g350"
I20260806 07:52:07.496309 11281 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:07.519582 11281 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:07.523793 11281 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:07.524111 11281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:07.546267 11281 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:38497
I20260806 07:52:07.546301 11348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:38497 every 8 connection(s)
I20260806 07:52:07.547923 11281 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:52:07.552446 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:52:07.554562 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11281
I20260806 07:52:07.554800 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:52:07.559827 11349 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002727, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:07.584116 11349 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002727, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:07.599022 11349 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002727, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:07.618331 11349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Bootstrap starting.
I20260806 07:52:07.621572 11349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Neither blocks nor log segments found. Creating new log.
I20260806 07:52:07.622912 11349 log.cc:826] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:07.625558 11349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: No bootstrap required, opened a new log
I20260806 07:52:07.631134 11349 raft_consensus.cc:359] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.631516 11349 raft_consensus.cc:385] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:52:07.631657 11349 raft_consensus.cc:740] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca65838c4315482791c4a8a1c9bee3aa, State: Initialized, Role: FOLLOWER
I20260806 07:52:07.632306 11349 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.633270 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.633514 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.634332 11349 sys_catalog.cc:565] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:52:07.645147 11371 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:52:07.645267 11371 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:52:07.642019 11281 master_runner.cc:187] Error getting master registration for 127.20.202.254:40139: OK, instance_id { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" instance_seqno: 1786002726338940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:52:07.647733 11281 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:52:07.670611 11271 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 5
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:07.689566 11271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Bootstrap starting.
I20260806 07:52:07.693674 11271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Neither blocks nor log segments found. Creating new log.
I20260806 07:52:07.694762 11271 log.cc:826] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:07.697314 11271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: No bootstrap required, opened a new log
I20260806 07:52:07.702831 11271 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.703092 11271 raft_consensus.cc:385] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:52:07.703266 11271 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b9f7ed0d5745e19c092f948a7b7d39, State: Initialized, Role: FOLLOWER
I20260806 07:52:07.704025 11271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.705205 11374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.705579 11374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.706524 11271 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:52:07.717010 11385 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:52:07.717224 11385 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:52:07.714130 11205 master_runner.cc:187] Error getting master registration for 127.20.202.254:40139: OK, instance_id { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" instance_seqno: 1786002726338940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:52:07.723759 11205 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:52:07.756738 11196 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } attempt: 5
Aug 06 07:52:07 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
W20260806 07:52:07.776371 11302 tablet.cc:2406] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:52:07.781265 11196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap starting.
I20260806 07:52:07.785277 11374 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:52:07.785544 11374 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.785771 11196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Neither blocks nor log segments found. Creating new log.
I20260806 07:52:07.787177 11196 log.cc:826] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:07.787392 11374 leader_election.cc:290] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c336a69c59f14d7da636e9a1b1fff164 (127.20.202.254:40139), ca65838c4315482791c4a8a1c9bee3aa (127.20.202.252:38497)
I20260806 07:52:07.788640 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca65838c4315482791c4a8a1c9bee3aa" is_pre_election: true
I20260806 07:52:07.789203 11324 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9f7ed0d5745e19c092f948a7b7d39 in term 0.
I20260806 07:52:07.789850 11222 leader_election.cc:304] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b9f7ed0d5745e19c092f948a7b7d39, ca65838c4315482791c4a8a1c9bee3aa; no voters:
I20260806 07:52:07.790604 11374 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:52:07.790755 11374 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:52:07.790856 11374 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:52:07.796172 11374 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.803505 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca65838c4315482791c4a8a1c9bee3aa"
I20260806 07:52:07.803725 11324 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:52:07.807680 11324 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b9f7ed0d5745e19c092f948a7b7d39 in term 1.
I20260806 07:52:07.808470 11222 leader_election.cc:304] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b9f7ed0d5745e19c092f948a7b7d39, ca65838c4315482791c4a8a1c9bee3aa; no voters:
I20260806 07:52:07.812765 11374 leader_election.cc:290] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [CANDIDATE]: Term 1 election: Requested vote from peers c336a69c59f14d7da636e9a1b1fff164 (127.20.202.254:40139), ca65838c4315482791c4a8a1c9bee3aa (127.20.202.252:38497)
I20260806 07:52:07.814038 11388 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:52:07.814319 11388 raft_consensus.cc:697] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 LEADER]: Becoming Leader. State: Replica: 28b9f7ed0d5745e19c092f948a7b7d39, State: Running, Role: LEADER
I20260806 07:52:07.815032 11388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.820963 11196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: No bootstrap required, opened a new log
I20260806 07:52:07.830935 11374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b9f7ed0d5745e19c092f948a7b7d39. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.831243 11374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:07.835825 11196 raft_consensus.cc:359] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.836563 11196 raft_consensus.cc:385] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:52:07.836825 11196 raft_consensus.cc:740] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c336a69c59f14d7da636e9a1b1fff164, State: Initialized, Role: FOLLOWER
I20260806 07:52:07.838826 11196 consensus_queue.cc:260] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:07.842357 11390 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:52:07.842548 11391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.842839 11391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.844815 11390 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:52:07.844895 11196 sys_catalog.cc:565] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:07.845062 11171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c336a69c59f14d7da636e9a1b1fff164" is_pre_election: true
I20260806 07:52:07.845674 11171 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b9f7ed0d5745e19c092f948a7b7d39 in term 0.
I20260806 07:52:07.844907 11170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c336a69c59f14d7da636e9a1b1fff164"
I20260806 07:52:07.846576 11170 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:52:07.850235 11170 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b9f7ed0d5745e19c092f948a7b7d39 in term 1.
I20260806 07:52:07.859448 11324 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 1 FOLLOWER]: Refusing update from remote peer 28b9f7ed0d5745e19c092f948a7b7d39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:52:07.861179 11396 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:52:07.866255 11170 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Refusing update from remote peer 28b9f7ed0d5745e19c092f948a7b7d39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:52:07.867280 11396 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:52:07.905997 11391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b9f7ed0d5745e19c092f948a7b7d39. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.906296 11391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.907879 11393 mvcc.cc:204] Tried to move back new op lower bound from 7315467173300273152 to 7315467173138718720. Current Snapshot: MvccSnapshot[applied={T|T < 7315467173300273152}]
I20260806 07:52:07.917757 11400 mvcc.cc:204] Tried to move back new op lower bound from 7315467173300273152 to 7315467173138718720. Current Snapshot: MvccSnapshot[applied={T|T < 7315467173300273152}]
I20260806 07:52:07.919698 11391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.919900 11391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.919620 11396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.920326 11396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:07.922379 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b9f7ed0d5745e19c092f948a7b7d39. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.922678 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.923847 11390 catalog_manager.cc:1383] Generated new cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:07.923952 11390 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:52:07.934434 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.934772 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:07.939432 11360 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:07.939704 11360 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:07.949802 11390 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:52:07.951561 11390 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:52:07.968562 11411 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:07.968732 11411 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: loading cluster ID for follower catalog manager: success
I20260806 07:52:07.972546 11411 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: acquiring CA information for follower catalog manager: success
W20260806 07:52:07.980118 11411 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 07:52:07.995324 11390 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Generated new TSK 0
I20260806 07:52:07.996492 11390 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:08.190068 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:08.611380 11419 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:08.611671 11419 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:08.611763 11419 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:08.620168 11419 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:08.620430 11419 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:08.631050 11419 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:08.632663 11419 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:08.634369 11419 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:08.644795 11424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:08.644847 11425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:08.648334 11371 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:08.648485 11371 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: loading cluster ID for follower catalog manager: success
W20260806 07:52:08.650841 11427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:08.652398 11419 server_base.cc:1061] running on GCE node
I20260806 07:52:08.653033 11419 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260806 07:52:08.653083 11371 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: acquiring CA information for follower catalog manager: success
W20260806 07:52:08.653211 11419 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:08.653309 11419 hybrid_clock.cc:648] HybridClock initialized: now 1786002728653287 us; error 0 us; skew 500 ppm
I20260806 07:52:08.653702 11419 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:52:08.655339 11371 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:52:08 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002728, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:08.657414 11419 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:08.659250 11419 webserver.cc:533] Webserver started at http://127.20.202.193:42965/ using document root <none> and password file <none>
I20260806 07:52:08.660017 11419 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:08.660122 11419 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:08.660490 11419 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:08.663018 11419 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:08.663800 11419 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:08.669481 11419 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:52:08.673202 11433 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:08.674777 11419 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:08.674917 11419 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:08.675143 11419 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:08.698449 11419 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:08.706219 11419 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:08.706540 11419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:08.707641 11419 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:08.709355 11419 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:08.709430 11419 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:08.709632 11419 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:08.709707 11419 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 07:52:08.748407 11419 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33361
I20260806 07:52:08.748546 11549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33361 every 8 connection(s)
I20260806 07:52:08.750204 11419 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:08.758026 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11419
I20260806 07:52:08.758325 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:52:08 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002728, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:08 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002728, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:08 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002728, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:08.794430 11552 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:08.794966 11552 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:08.796399 11552 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:08.798297 11551 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:08.798549 11551 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:08.799161 11551 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:08.799712 11236 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:33361)
I20260806 07:52:08.801260 11159 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:33361)
I20260806 07:52:08.802341 11236 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:47569
I20260806 07:52:08.805430 11550 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:08.805567 11550 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:08.805886 11550 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:08.807750 11313 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:33361)
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:08.902240 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:08.992288 11411 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:52:09.257306 11564 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:09.257720 11564 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:09.257884 11564 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:09.266496 11564 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:09.266752 11564 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:52:09.277810 11564 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:09.279636 11564 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:09.281334 11564 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:09.291421 11572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:09.291392 11570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:09.292550 11574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:09.292857 11564 server_base.cc:1061] running on GCE node
I20260806 07:52:09.293437 11564 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:09.293664 11564 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:09.293771 11564 hybrid_clock.cc:648] HybridClock initialized: now 1786002729293748 us; error 0 us; skew 500 ppm
I20260806 07:52:09.294077 11564 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:09 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:09.297703 11564 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:52:09.299383 11564 webserver.cc:533] Webserver started at http://127.20.202.194:42237/ using document root <none> and password file <none>
I20260806 07:52:09.300177 11564 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:09.300302 11564 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:09.300654 11564 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:09.303468 11564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "15b7eb849fd54f218b9954353394c000"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:09.304361 11564 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "15b7eb849fd54f218b9954353394c000"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:09.310452 11564 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 07:52:09.314046 11580 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:09.315488 11564 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:52:09.315707 11564 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "15b7eb849fd54f218b9954353394c000"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:09.315886 11564 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:09.337212 11564 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:09.341518 11564 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:09.341915 11564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:09.342952 11564 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:09.344735 11564 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:09.344852 11564 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:09.345021 11564 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:09.345094 11564 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:09.383050 11564 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:39319
I20260806 07:52:09.383136 11695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:39319 every 8 connection(s)
I20260806 07:52:09.384501 11564 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:52:09.390962 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11564
I20260806 07:52:09.391256 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:52:09 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:09 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:09 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:09.437172 11696 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:09.437325 11698 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:09.437702 11696 heartbeater.cc:461] Registering TS with master...
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
I20260806 07:52:09.438876 11696 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:09.437736 11697 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:09.439412 11697 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:09.437698 11698 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:09.440133 11697 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:09.440569 11698 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:09.441113 11313 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:39319)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.195@KRBTEST.COM" created.
I20260806 07:52:09.441514 11159 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:39319)
I20260806 07:52:09.441560 11236 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:39319)
I20260806 07:52:09.442502 11236 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:56325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:09.527101 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:09.805742 11552 heartbeater.cc:499] Master 127.20.202.253:38221 was elected leader, sending a full tablet report...
W20260806 07:52:09.892648 11710 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:09.893043 11710 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:09.893208 11710 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:09.903057 11710 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:09.903285 11710 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:52:09.914324 11710 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:09.916087 11710 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:09.917486 11710 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:09.928128 11716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:09.928165 11718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:09.928079 11715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:09.929872 11710 server_base.cc:1061] running on GCE node
I20260806 07:52:09.930469 11710 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:09.930653 11710 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:09.930763 11710 hybrid_clock.cc:648] HybridClock initialized: now 1786002729930721 us; error 0 us; skew 500 ppm
I20260806 07:52:09.931062 11710 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:09 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:09.934437 11710 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:52:09.936125 11710 webserver.cc:533] Webserver started at http://127.20.202.195:35649/ using document root <none> and password file <none>
I20260806 07:52:09.937029 11710 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:09.937129 11710 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:09.937518 11710 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:09.941037 11710 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "0d73f52291d14582b9e0116d578a7603"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:09.942278 11710 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "0d73f52291d14582b9e0116d578a7603"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:09.948748 11710 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 07:52:09.952373 11725 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:09.954030 11710 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:09.954195 11710 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "0d73f52291d14582b9e0116d578a7603"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:09.954367 11710 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:09.979966 11710 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:09.984398 11710 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:09.984721 11710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:09.985864 11710 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:09.987519 11710 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:09.987634 11710 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:09.987792 11710 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:09.987866 11710 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:10.026469 11710 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:34845
I20260806 07:52:10.026556 11838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:34845 every 8 connection(s)
I20260806 07:52:10.027959 11710 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:52:10.035982 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11710
I20260806 07:52:10.036248 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:52:10 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:10 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:10 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002729, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:10.070537 11841 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:10.070606 11839 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:10.071017 11839 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:10.071022 11841 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:10.072270 11839 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:10.072293 11841 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:10.072453 11840 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:10.072599 11840 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:10.073007 11840 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:10.074466 11159 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:34845)
I20260806 07:52:10.074587 11313 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:34845)
I20260806 07:52:10.075188 11236 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:34845)
I20260806 07:52:10.076228 11236 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:44209
I20260806 07:52:10.083547 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:52:10.091068 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:52:10.091323 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11129
I20260806 07:52:10.092366 11191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:10.201828 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11129
I20260806 07:52:10.225456 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--webserver_interface=127.20.202.254
--webserver_port=35483
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:10.444531 11697 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:40139 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:40139: connect: Connection refused (error 111)
I20260806 07:52:10.444818 11698 heartbeater.cc:499] Master 127.20.202.253:38221 was elected leader, sending a full tablet report...
W20260806 07:52:10.474474 11222 consensus_peers.cc:597] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 -> Peer c336a69c59f14d7da636e9a1b1fff164 (127.20.202.254:40139): Couldn't send request to peer c336a69c59f14d7da636e9a1b1fff164. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:40139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:52:10.579145 11850 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:10.579567 11850 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:10.579728 11850 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:10.588307 11850 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:10.588405 11850 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:10.588459 11850 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:10.588506 11850 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:10.599373 11850 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=35483
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:10.601039 11850 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:10.602430 11850 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:10.611112 11860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:10.611176 11861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:10.611155 11863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:10.612794 11850 server_base.cc:1061] running on GCE node
I20260806 07:52:10.613505 11850 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:10.613716 11850 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:10.613792 11850 hybrid_clock.cc:648] HybridClock initialized: now 1786002730613774 us; error 0 us; skew 500 ppm
I20260806 07:52:10.614099 11850 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:10 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002730, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:10.618896 11850 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:52:10.621762 11850 webserver.cc:533] Webserver started at http://127.20.202.254:35483/ using document root <none> and password file <none>
I20260806 07:52:10.622789 11850 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:10.622948 11850 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:10.628993 11850 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:52:10.631891 11869 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:10.633272 11850 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:52:10.633441 11850 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c336a69c59f14d7da636e9a1b1fff164"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:10.634295 11850 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:10.650013 11850 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:10.654212 11850 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:10.654583 11850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:10.676303 11850 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40139
I20260806 07:52:10.676270 11920 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40139 every 8 connection(s)
I20260806 07:52:10.678023 11850 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:52:10.682180 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11850
I20260806 07:52:10.682530 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11419
I20260806 07:52:10.686197 11921 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:52:10.689689 11921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap starting.
Aug 06 07:52:10 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:52:10.700939 11545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:10.711408 11921 log.cc:826] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:10.723520 11921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:52:10.724198 11921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap complete.
I20260806 07:52:10.734596 11921 raft_consensus.cc:359] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:10.735707 11921 raft_consensus.cc:740] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c336a69c59f14d7da636e9a1b1fff164, State: Initialized, Role: FOLLOWER
I20260806 07:52:10.736666 11921 consensus_queue.cc:260] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:10.737711 11929 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:10.738034 11929 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:10.738842 11921 sys_catalog.cc:565] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:10.749464 11940 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:10.749576 11940 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: loading cluster ID for follower catalog manager: success
I20260806 07:52:10.754142 11940 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: acquiring CA information for follower catalog manager: success
I20260806 07:52:10.756759 11940 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:52:10.821547 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11419
W20260806 07:52:10.845989 11297 connection.cc:573] server connection from 127.20.202.193:48441 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:52:10.847496 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33361
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--webserver_port=42965
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:10.847574 11943 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:10.815712 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:53457 (local address 127.20.202.254:40139)
0806 07:52:10.815904 (+ 192us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:10.815911 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:10.821473 (+ 5562us) server_negotiation.cc:408] Connection header received
0806 07:52:10.821931 (+ 458us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:10.821937 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:10.822023 (+ 86us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:10.822090 (+ 67us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0806 07:52:10.846091 (+ 24001us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:53457: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 07:52:11.066957 11852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:11.067271 11852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:11.079658 11841 heartbeater.cc:499] Master 127.20.202.253:38221 was elected leader, sending a full tablet report...
I20260806 07:52:11.083841 11929 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b9f7ed0d5745e19c092f948a7b7d39. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:11.084131 11929 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:11.103695 11886 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "0d73f52291d14582b9e0116d578a7603" instance_seqno: 1786002730017368) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:38209; Asking this server to re-register.
I20260806 07:52:11.104748 11840 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:11.106669 11886 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:34845)
W20260806 07:52:11.215842 11947 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:11.216223 11947 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:11.216395 11947 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:11.227798 11947 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:11.228053 11947 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:11.239202 11947 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42965
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:11.241086 11947 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:11.242591 11947 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:11.252830 11959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:11.252959 11958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:11.252848 11961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:11.254410 11947 server_base.cc:1061] running on GCE node
I20260806 07:52:11.254957 11947 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:11.255093 11947 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:11.255157 11947 hybrid_clock.cc:648] HybridClock initialized: now 1786002731255140 us; error 0 us; skew 500 ppm
I20260806 07:52:11.255394 11947 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:11 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:11.259038 11947 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:11.260733 11947 webserver.cc:533] Webserver started at http://127.20.202.193:42965/ using document root <none> and password file <none>
I20260806 07:52:11.261503 11947 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:11.261700 11947 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:11.267202 11947 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:52:11.270085 11967 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:11.271502 11947 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:52:11.271694 11947 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:11.272425 11947 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:11.287988 11947 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:11.292311 11947 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:11.292588 11947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:11.293565 11947 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:11.295193 11947 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:11.295370 11947 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:11.295492 11947 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:11.295576 11947 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:11.330986 11947 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33361
I20260806 07:52:11.331105 12081 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33361 every 8 connection(s)
I20260806 07:52:11.332722 11947 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:11.336665 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 11947
Aug 06 07:52:11 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:52:11 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:11 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:11 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:11.370538 12084 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:11.370988 12084 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:11.371991 12084 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:11.372470 12083 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:11.372641 12083 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:11.373274 12083 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:11.374239 11236 ts_manager.cc:194] Re-registered known tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:33361)
I20260806 07:52:11.374471 11886 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:33361)
I20260806 07:52:11.375351 11236 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:45665
I20260806 07:52:11.380179 12082 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:11.380333 12082 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:11.380653 12082 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:11.381814 11313 ts_manager.cc:194] Re-registered known tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:33361)
I20260806 07:52:11.471045 11697 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:11.472283 11886 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "15b7eb849fd54f218b9954353394c000" instance_seqno: 1786002729374143) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:50973; Asking this server to re-register.
I20260806 07:52:11.472862 11697 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:11.474045 11886 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:39319)
I20260806 07:52:11.479979 11949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:11.480366 11949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:11.507810 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11947
W20260806 07:52:11.509095 12097 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:11.505327 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.20.202.193:33361)
0806 07:52:11.506095 (+ 768us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:11.506103 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:11.506134 (+ 31us) server_negotiation.cc:408] Connection header received
0806 07:52:11.506196 (+ 62us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:11.506203 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:11.506411 (+ 208us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:11.506575 (+ 164us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:11.507495 (+ 920us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34412: BlockingRecv error: recv got EOF from 127.0.0.1:34412 (error 108)
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":442,"threads_started":1}
W20260806 07:52:11.510408 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11947: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33361: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:11.510576 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11947
W20260806 07:52:11.510663 12098 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:11.508600 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.20.202.193:33361)
0806 07:52:11.509314 (+ 714us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:11.509320 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:11.509342 (+ 22us) server_negotiation.cc:408] Connection header received
0806 07:52:11.509402 (+ 60us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:11.509406 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:11.509458 (+ 52us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:11.509610 (+ 152us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:11.510393 (+ 783us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34418: BlockingRecv error: recv got EOF from 127.0.0.1:34418 (error 108)
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":437,"threads_started":1}
I20260806 07:52:11.529515 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33361
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--webserver_port=42965
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:11.881104 12099 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:11.881379 12099 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:11.881510 12099 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:52:11.881640 12099 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:11.889901 12099 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:11.890161 12099 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:11.900652 12099 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42965
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:11.902420 12099 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:11.903832 12099 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:11.913992 12104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:11.914340 12105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:11.914677 12107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:11.915522 12099 server_base.cc:1061] running on GCE node
I20260806 07:52:11.916147 12099 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:11.916307 12099 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:11.916435 12099 hybrid_clock.cc:648] HybridClock initialized: now 1786002731916381 us; error 0 us; skew 500 ppm
I20260806 07:52:11.916740 12099 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:11 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:11.920176 12099 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:11.922080 12099 webserver.cc:533] Webserver started at http://127.20.202.193:42965/ using document root <none> and password file <none>
I20260806 07:52:11.922887 12099 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:11.923003 12099 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:11.928666 12099 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:52:11.931551 12113 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:11.932932 12099 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:52:11.933128 12099 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:11.933912 12099 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:11.952221 12099 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:11.958127 12099 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:11.958451 12099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:11.959535 12099 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:11.963986 12099 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:11.964067 12099 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:11.964196 12099 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:11.964270 12099 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:52:11.964752 12099 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:52:11.964841 12099 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:52:11.964947 12099 fault_injection.cc:59] injecting fault for kudu (pid 12099): FLAGS_fault_before_start (process will exit)
I20260806 07:52:11.988416 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33361
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--webserver_port=42965
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:12.334535 12121 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:12.334857 12121 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:12.335053 12121 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:12.344048 12121 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:12.344285 12121 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:12.355091 12121 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42965
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:12.356837 12121 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:12.358325 12121 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:12.368634 12127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:12.368598 12126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:12.369570 12129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:12.370085 12121 server_base.cc:1061] running on GCE node
I20260806 07:52:12.370675 12121 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:12.370862 12121 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:12.370970 12121 hybrid_clock.cc:648] HybridClock initialized: now 1786002732370930 us; error 0 us; skew 500 ppm
I20260806 07:52:12.371294 12121 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:12 dist-test-slave-g350 krb5kdc[10830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:12.374713 12121 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:12.376343 12121 webserver.cc:533] Webserver started at http://127.20.202.193:42965/ using document root <none> and password file <none>
I20260806 07:52:12.377121 12121 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:12.377262 12121 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:12.383065 12121 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:52:12.386394 12135 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:12.387789 12121 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:52:12.387957 12121 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:12.388695 12121 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:12.401535 12121 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:12.406109 12121 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:12.406412 12121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:12.407545 12121 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:12.409288 12121 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:12.409386 12121 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:12.409510 12121 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:12.409668 12121 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 07:52:12.446770 12121 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33361
I20260806 07:52:12.446810 12249 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33361 every 8 connection(s)
I20260806 07:52:12.448433 12121 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:12.457248 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 12121
I20260806 07:52:12.457711 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12121
Aug 06 07:52:12 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:12 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:12 dist-test-slave-g350 krb5kdc[10830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:12.482414 12250 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:12.482825 12250 heartbeater.cc:461] Registering TS with master...
W20260806 07:52:12.483495 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12121: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33361: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:12.483692 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 12121
I20260806 07:52:12.499291 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11564
W20260806 07:52:12.500571 12254 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:12.451928 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:36199 (local address 127.20.202.253:38221)
0806 07:52:12.452430 (+ 502us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:12.452437 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:12.453872 (+ 1435us) server_negotiation.cc:408] Connection header received
0806 07:52:12.456666 (+ 2794us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:12.456673 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:12.456772 (+ 99us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:12.456885 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:12.462215 (+ 5330us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:52:12.463696 (+ 1481us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:52:12.472783 (+ 9087us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:52:12.473428 (+ 645us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:52:12.477849 (+ 4421us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0806 07:52:12.477905 (+ 56us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0806 07:52:12.477911 (+ 6us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
0806 07:52:12.477970 (+ 59us) server_negotiation.cc:923] calling sasl_server_start()
0806 07:52:12.483173 (+ 5203us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
0806 07:52:12.499294 (+ 16121us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:36199: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":402,"threads_started":1}
W20260806 07:52:12.502560 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11564: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:39319: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:12.502708 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11564
I20260806 07:52:12.520226 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11710
W20260806 07:52:12.523077 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11710: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:34845: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:12.523262 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11710
I20260806 07:52:12.538256 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11850
W20260806 07:52:12.540911 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11850: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:40139: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:12.541059 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11850
W20260806 07:52:12.541088 12255 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:12.539343 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.20.202.254:40139)
0806 07:52:12.539552 (+ 209us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:12.539558 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:12.539579 (+ 21us) server_negotiation.cc:408] Connection header received
0806 07:52:12.539845 (+ 266us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:12.539851 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:12.539920 (+ 69us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:12.540031 (+ 111us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:12.540800 (+ 769us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34610: BlockingRecv error: recv got EOF from 127.0.0.1:34610 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 07:52:12.563519 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11205
I20260806 07:52:12.564831 11266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:12.649732 11281 master_runner.cc:187] Error getting master registration for 127.20.202.254:40139: Network error: Client connection negotiation failed: client connection to 127.20.202.254:40139: connect: Connection refused (error 111),
I20260806 07:52:12.649983 11281 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:52:12.677834 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11205
I20260806 07:52:12.700675 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 11281
I20260806 07:52:12.701972 11344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:12.816283 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 11281
Shutting down hive metastore.
Aug 06 07:52:13 dist-test-slave-g350 krb5kdc[12273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:52:13 dist-test-slave-g350 krb5kdc[12273](info): set up 2 sockets
Aug 06 07:52:13 dist-test-slave-g350 krb5kdc[12273](info): commencing operation
krb5kdc: starting...
W20260806 07:52:15.234963 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.015s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:52:15 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "hive/127.0.0.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:52:16: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:52:18 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002738, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:52:23 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002743, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:52:23 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002743, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:24.914158 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:25.265407 12468 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:25.265849 12468 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:25.266007 12468 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:25.274811 12468 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:25.274987 12468 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:25.275113 12468 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:25.275197 12468 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:25.286404 12468 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:25.288152 12468 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:25.289564 12468 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:25.298385 12474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:25.298385 12473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:25.298754 12476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:25.299767 12468 server_base.cc:1061] running on GCE node
I20260806 07:52:25.300640 12468 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:25.300832 12468 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:25.300930 12468 hybrid_clock.cc:648] HybridClock initialized: now 1786002745300889 us; error 0 us; skew 500 ppm
I20260806 07:52:25.301311 12468 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:25 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:25.305123 12468 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:52:25.307128 12468 webserver.cc:533] Webserver started at http://127.20.202.254:44005/ using document root <none> and password file <none>
I20260806 07:52:25.307821 12468 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:25.307915 12468 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:25.313472 12468 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:52:25.316471 12482 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:25.318101 12468 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:25.318290 12468 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c336a69c59f14d7da636e9a1b1fff164"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:25.319041 12468 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:25.346679 12468 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:25.351284 12468 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:25.351594 12468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:25.374548 12468 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40139
I20260806 07:52:25.374599 12534 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40139 every 8 connection(s)
I20260806 07:52:25.376328 12468 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:52:25.383447 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 12468
I20260806 07:52:25.383702 12535 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:52:25.383730 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:52:25.387200 12535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap starting.
I20260806 07:52:25.407621 12535 log.cc:826] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:52:25.418438 12535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:52:25.418974 12535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap complete.
I20260806 07:52:25.429546 12535 raft_consensus.cc:359] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:25.430707 12535 raft_consensus.cc:740] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c336a69c59f14d7da636e9a1b1fff164, State: Initialized, Role: FOLLOWER
I20260806 07:52:25.431648 12535 consensus_queue.cc:260] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:25.432566 12541 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:25.432864 12541 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:25.433966 12535 sys_catalog.cc:565] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:25.445560 12552 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:25.445775 12552 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: loading cluster ID for follower catalog manager: success
I20260806 07:52:25.450114 12552 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: acquiring CA information for follower catalog manager: success
I20260806 07:52:25.453263 12552 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:25.469916 12468 master_runner.cc:187] Error getting master registration for 127.20.202.253:38221: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38221: connect: Connection refused (error 111),
I20260806 07:52:25.474680 12468 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:25.521654 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38221
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:25.893157 12558 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:25.893651 12558 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:25.893841 12558 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:25.903079 12558 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:25.903250 12558 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:25.903371 12558 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:25.903465 12558 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:25.915823 12558 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:25.917726 12558 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:25.919278 12558 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:25.928385 12563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:25.928733 12564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:25.929342 12566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:25.930109 12558 server_base.cc:1061] running on GCE node
I20260806 07:52:25.930946 12558 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:25.931154 12558 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:25.931236 12558 hybrid_clock.cc:648] HybridClock initialized: now 1786002745931217 us; error 0 us; skew 500 ppm
I20260806 07:52:25.931562 12558 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:25 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:25.935575 12558 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:52:25.937701 12558 webserver.cc:533] Webserver started at http://127.20.202.253:33443/ using document root <none> and password file <none>
I20260806 07:52:25.938573 12558 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:25.938683 12558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:25.947077 12558 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 07:52:25.950937 12572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:25.952742 12558 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:52:25.952967 12558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "28b9f7ed0d5745e19c092f948a7b7d39"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:25.953886 12558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:25.970860 12558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:25.975155 12558 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:25.975447 12558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:25.997799 12558 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:38221
I20260806 07:52:25.997774 12623 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:38221 every 8 connection(s)
I20260806 07:52:25.999567 12558 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:52:26.002676 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 12558
I20260806 07:52:26.002918 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:52:26.007210 12624 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:52:26.010944 12624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Bootstrap starting.
I20260806 07:52:26.033803 12624 log.cc:826] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:52:26.043574 12624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:52:26.044169 12624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: Bootstrap complete.
I20260806 07:52:26.056093 12624 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:26.057314 12624 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b9f7ed0d5745e19c092f948a7b7d39, State: Initialized, Role: FOLLOWER
I20260806 07:52:26.058415 12624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:26.059576 12629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:26.060097 12629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:26.061138 12624 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:26.073329 12640 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:26.073463 12640 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: loading cluster ID for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:26.078699 12640 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: acquiring CA information for follower catalog manager: success
I20260806 07:52:26.082099 12640 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:52:26.113571 12558 master_runner.cc:187] Error getting master registration for 127.20.202.252:38497: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38497: connect: Connection refused (error 111),
I20260806 07:52:26.117730 12558 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:26.154433 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38497
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:26.537456 12648 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:26.537915 12648 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:26.538021 12648 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:26.546512 12648 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:26.546622 12648 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:26.546679 12648 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:26.546727 12648 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:26.557717 12648 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:26.559495 12648 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:26.561077 12648 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:26.570111 12653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:26.570106 12654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:26.571141 12656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:26.572387 12648 server_base.cc:1061] running on GCE node
I20260806 07:52:26.573241 12648 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:26.573453 12648 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:26.573550 12648 hybrid_clock.cc:648] HybridClock initialized: now 1786002746573511 us; error 0 us; skew 500 ppm
I20260806 07:52:26.573925 12648 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:26.577823 12648 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:52:26.579941 12648 webserver.cc:533] Webserver started at http://127.20.202.252:43007/ using document root <none> and password file <none>
I20260806 07:52:26.580689 12648 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:26.580798 12648 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:26.586911 12648 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:52:26.590144 12662 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:26.591714 12648 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:52:26.591908 12648 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "ca65838c4315482791c4a8a1c9bee3aa"
format_stamp: "Formatted at 2026-08-06 07:52:07 on dist-test-slave-g350"
I20260806 07:52:26.592682 12648 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:26.610504 12648 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:26.615039 12648 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:26.615335 12648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:26.637547 12648 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:38497
I20260806 07:52:26.637578 12713 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:38497 every 8 connection(s)
I20260806 07:52:26.639453 12648 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:52:26.646772 12714 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:52:26.649305 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 12648
I20260806 07:52:26.649524 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:52:26.650424 12714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Bootstrap starting.
I20260806 07:52:26.669250 12720 raft_consensus.cc:493] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:52:26.669683 12720 raft_consensus.cc:515] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:26.676689 12714 log.cc:826] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:26.679920 12720 leader_election.cc:290] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28b9f7ed0d5745e19c092f948a7b7d39 (127.20.202.253:38221), ca65838c4315482791c4a8a1c9bee3aa (127.20.202.252:38497)
I20260806 07:52:26.692528 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c336a69c59f14d7da636e9a1b1fff164" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" is_pre_election: true
I20260806 07:52:26.693174 12714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:52:26.693252 12599 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c336a69c59f14d7da636e9a1b1fff164 in term 1.
I20260806 07:52:26.693878 12714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Bootstrap complete.
I20260806 07:52:26.694370 12484 leader_election.cc:304] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b9f7ed0d5745e19c092f948a7b7d39, c336a69c59f14d7da636e9a1b1fff164; no voters:
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:26.695626 12720 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:52:26.695871 12720 raft_consensus.cc:493] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:52:26.696012 12720 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 1 FOLLOWER]: Advancing to term 2
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:26.703758 12720 raft_consensus.cc:515] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:26.703912 12714 raft_consensus.cc:359] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:26.705101 12714 raft_consensus.cc:740] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca65838c4315482791c4a8a1c9bee3aa, State: Initialized, Role: FOLLOWER
I20260806 07:52:26.705410 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c336a69c59f14d7da636e9a1b1fff164" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28b9f7ed0d5745e19c092f948a7b7d39"
I20260806 07:52:26.706140 12714 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:26.706493 12599 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:52:26.712430 12599 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c336a69c59f14d7da636e9a1b1fff164 in term 2.
I20260806 07:52:26.715538 12484 leader_election.cc:304] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b9f7ed0d5745e19c092f948a7b7d39, c336a69c59f14d7da636e9a1b1fff164; no voters:
I20260806 07:52:26.716061 12720 leader_election.cc:290] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [CANDIDATE]: Term 2 election: Requested vote from peers 28b9f7ed0d5745e19c092f948a7b7d39 (127.20.202.253:38221), ca65838c4315482791c4a8a1c9bee3aa (127.20.202.252:38497)
I20260806 07:52:26.716728 12720 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:52:26.717206 12720 raft_consensus.cc:697] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 LEADER]: Becoming Leader. State: Replica: c336a69c59f14d7da636e9a1b1fff164, State: Running, Role: LEADER
I20260806 07:52:26.718534 12720 consensus_queue.cc:237] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:26.724718 12726 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:26.726106 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c336a69c59f14d7da636e9a1b1fff164. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:26.726475 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:26.726820 12726 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:26.728219 12729 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:52:26.728803 12714 sys_catalog.cc:565] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:26.728883 12689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c336a69c59f14d7da636e9a1b1fff164" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca65838c4315482791c4a8a1c9bee3aa" is_pre_election: true
I20260806 07:52:26.729831 12688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c336a69c59f14d7da636e9a1b1fff164" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ca65838c4315482791c4a8a1c9bee3aa"
I20260806 07:52:26.730415 12689 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c336a69c59f14d7da636e9a1b1fff164 in term 1.
I20260806 07:52:26.731087 12729 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:52:26.732573 12729 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:26.732685 12729 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:52:26.734774 12729 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:52:26.736672 12729 catalog_manager.cc:6081] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Loaded TSK: 0
I20260806 07:52:26.738315 12729 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:52:26.745779 12741 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:26.745919 12741 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: loading cluster ID for follower catalog manager: success
I20260806 07:52:26.749303 12741 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: acquiring CA information for follower catalog manager: success
Aug 06 07:52:26 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:26.754532 12741 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:26.897339 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:27.115437 12688 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:52:27.122761 12688 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 2 FOLLOWER]: Refusing update from remote peer c336a69c59f14d7da636e9a1b1fff164: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:52:27.124141 12727 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:52:27.128796 12726 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c336a69c59f14d7da636e9a1b1fff164. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:27.129089 12726 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:27.131183 12727 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:27.131525 12727 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:27.133342 12599 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 2 FOLLOWER]: Refusing update from remote peer c336a69c59f14d7da636e9a1b1fff164: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:52:27.134438 12727 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:52:27.147679 12726 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:27.147949 12726 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:27.152823 12752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c336a69c59f14d7da636e9a1b1fff164. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:27.153291 12752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:27.153321 12756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:27.153821 12756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:52:27.276593 12748 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:27.276959 12748 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:27.277057 12748 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:27.285928 12748 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:27.286113 12748 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:27.297127 12748 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:27.298908 12748 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:27.300289 12748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:27.310254 12762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:27.310546 12761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:27.310961 12764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:27.311883 12748 server_base.cc:1061] running on GCE node
I20260806 07:52:27.312496 12748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:27.312644 12748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:27.312712 12748 hybrid_clock.cc:648] HybridClock initialized: now 1786002747312694 us; error 0 us; skew 500 ppm
I20260806 07:52:27.313017 12748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:27 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:27.316874 12748 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:27.318699 12748 webserver.cc:533] Webserver started at http://127.20.202.193:44533/ using document root <none> and password file <none>
I20260806 07:52:27.319478 12748 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:27.319644 12748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:27.325663 12748 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:52:27.328595 12770 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:27.330224 12748 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:52:27.330415 12748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:27.331243 12748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:27.347891 12748 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:27.352268 12748 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:27.352608 12748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:27.353731 12748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:27.355420 12748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:27.355548 12748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:27.355716 12748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:27.355805 12748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:27.393241 12748 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35563
I20260806 07:52:27.393299 12884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35563 every 8 connection(s)
I20260806 07:52:27.394912 12748 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:27.397785 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 12748
I20260806 07:52:27.398072 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:52:27 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:27 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:27 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:27.439306 12887 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:27.439337 12886 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:27.439765 12886 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:27.439770 12887 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:27.440855 12887 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:27.440917 12886 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:27.441387 12885 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:27.441526 12885 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:27.442078 12885 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:27.443329 12499 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:35563)
I20260806 07:52:27.443392 12589 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:35563)
I20260806 07:52:27.444839 12679 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:35563)
I20260806 07:52:27.445541 12499 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:39643
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:27.541582 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:27.673233 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c336a69c59f14d7da636e9a1b1fff164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:27.673532 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: LEADER
W20260806 07:52:27.900861 12899 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:27.901177 12899 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:27.901381 12899 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:27.910212 12899 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:27.910449 12899 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:52:27.921970 12899 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:27.923729 12899 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:27.925146 12899 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:27.947844 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:27.948750 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:27.948889 12910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:27.950026 12899 server_base.cc:1061] running on GCE node
I20260806 07:52:27.950888 12899 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:27.951109 12899 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:27.951220 12899 hybrid_clock.cc:648] HybridClock initialized: now 1786002747951198 us; error 0 us; skew 500 ppm
I20260806 07:52:27.951617 12899 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:27 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:27.955731 12899 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:52:27.957739 12899 webserver.cc:533] Webserver started at http://127.20.202.194:37009/ using document root <none> and password file <none>
I20260806 07:52:27.958578 12899 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:27.958747 12899 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:27.964814 12899 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:52:27.967927 12916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:27.969377 12899 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:52:27.969671 12899 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "15b7eb849fd54f218b9954353394c000"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:27.970443 12899 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:27.984195 12899 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:27.988821 12899 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:27.989151 12899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:27.990327 12899 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:27.992098 12899 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:27.992234 12899 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:27.992385 12899 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:27.992502 12899 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:28.037861 12899 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:44657
I20260806 07:52:28.037925 13030 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:44657 every 8 connection(s)
I20260806 07:52:28.039584 12899 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:52:28.041221 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 12899
I20260806 07:52:28.041429 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002747, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:28.085107 13031 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:28.085263 13033 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:28.085580 13031 heartbeater.cc:461] Registering TS with master...
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
I20260806 07:52:28.085621 13033 heartbeater.cc:461] Registering TS with master...
add_principal: Principal or policy already exists while creating "kudu/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:52:28.086725 13032 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:28.086905 13032 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:28.087088 13033 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:28.087303 13031 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:28.087612 13032 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:28.089386 12499 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:44657)
I20260806 07:52:28.089618 12589 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:44657)
I20260806 07:52:28.090139 12679 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:44657)
I20260806 07:52:28.090703 12499 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:40929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:28.181164 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:28.448251 12886 heartbeater.cc:499] Master 127.20.202.254:40139 was elected leader, sending a full tablet report...
W20260806 07:52:28.544576 13045 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:28.544912 13045 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:28.545109 13045 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:28.554775 13045 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:28.555049 13045 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:52:28.566401 13045 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:28.568205 13045 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:28.569661 13045 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:28.579494 13050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:28.579836 13051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:28.580279 13053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:28.581453 13045 server_base.cc:1061] running on GCE node
I20260806 07:52:28.582146 13045 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:28.582322 13045 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:28.582437 13045 hybrid_clock.cc:648] HybridClock initialized: now 1786002748582415 us; error 0 us; skew 500 ppm
I20260806 07:52:28.582721 13045 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002748, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:28.586752 13045 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:52:28.588532 13045 webserver.cc:533] Webserver started at http://127.20.202.195:43587/ using document root <none> and password file <none>
I20260806 07:52:28.589277 13045 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:28.589387 13045 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:28.595052 13045 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:52:28.597923 13059 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:28.599314 13045 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:28.599457 13045 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "0d73f52291d14582b9e0116d578a7603"
format_stamp: "Formatted at 2026-08-06 07:52:09 on dist-test-slave-g350"
I20260806 07:52:28.600199 13045 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:28.618918 13045 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:28.623368 13045 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:28.623605 13045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:28.624689 13045 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:28.626350 13045 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:28.626438 13045 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:28.626528 13045 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:28.626590 13045 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:28.664240 13045 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:40493
I20260806 07:52:28.664290 13172 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:40493 every 8 connection(s)
I20260806 07:52:28.666129 13045 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:52:28.671681 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 13045
I20260806 07:52:28.671964 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002748, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002748, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:28 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002748, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:28.705345 13175 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:28.705823 13175 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:28.706452 13173 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:28.706619 13173 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:28.706985 13175 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:28.707218 13173 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:28.707336 13174 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:28.707489 13174 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:28.708051 13174 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:28.709082 12679 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:40493)
I20260806 07:52:28.709082 12589 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:40493)
I20260806 07:52:28.709287 12499 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:40493)
I20260806 07:52:28.710471 12499 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:59937
I20260806 07:52:28.714488 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:52:28.722929 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:52:28.723143 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12468
I20260806 07:52:28.724157 12529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:28.838541 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 12468
W20260806 07:52:28.863468 12573 connection.cc:573] server connection from 127.0.0.1:38392 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:52:28.864369 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--webserver_interface=127.20.202.254
--webserver_port=44005
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:29.093708 13032 heartbeater.cc:499] Master 127.20.202.254:40139 was elected leader, sending a full tablet report...
W20260806 07:52:29.095326 13032 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:40139 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:40139: connect: Connection refused (error 111)
W20260806 07:52:29.241814 13185 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:29.242236 13185 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:29.242394 13185 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:29.251050 13185 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:29.251223 13185 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:29.251351 13185 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:29.251434 13185 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:29.263089 13185 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=44005
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:29.264988 13185 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:29.266669 13185 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:29.276176 13191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:29.276170 13192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:29.276611 13194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:29.277017 13185 server_base.cc:1061] running on GCE node
I20260806 07:52:29.277987 13185 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:29.278195 13185 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:29.278307 13185 hybrid_clock.cc:648] HybridClock initialized: now 1786002749278284 us; error 0 us; skew 500 ppm
I20260806 07:52:29.278703 13185 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:29 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002749, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:29.282970 13185 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:52:29.285207 13185 webserver.cc:533] Webserver started at http://127.20.202.254:44005/ using document root <none> and password file <none>
I20260806 07:52:29.286080 13185 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:29.286231 13185 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:29.292248 13185 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 07:52:29.295547 13200 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:29.297253 13185 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:52:29.297482 13185 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c336a69c59f14d7da636e9a1b1fff164"
format_stamp: "Formatted at 2026-08-06 07:52:06 on dist-test-slave-g350"
I20260806 07:52:29.298456 13185 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:29.321458 13185 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:29.331144 13185 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:29.331441 13185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:29.354300 13185 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40139
I20260806 07:52:29.354323 13252 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40139 every 8 connection(s)
I20260806 07:52:29.356088 13185 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:52:29.363749 13253 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:52:29.365805 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 13185
I20260806 07:52:29.366142 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12748
I20260806 07:52:29.368263 13253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap starting.
Aug 06 07:52:29 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:52:29.387542 12880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:29.388453 13253 log.cc:826] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:29.399282 13253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:52:29.399866 13253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: Bootstrap complete.
I20260806 07:52:29.407346 13253 raft_consensus.cc:359] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:29.408106 13253 raft_consensus.cc:740] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c336a69c59f14d7da636e9a1b1fff164, State: Initialized, Role: FOLLOWER
I20260806 07:52:29.408866 13253 consensus_queue.cc:260] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:29.409826 13259 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:29.410144 13259 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:29.410885 13253 sys_catalog.cc:565] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:29.433491 13270 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:29.433740 13270 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: loading cluster ID for follower catalog manager: success
I20260806 07:52:29.438651 13270 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: acquiring CA information for follower catalog manager: success
I20260806 07:52:29.442096 13270 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:52:29 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002749, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:29.482404 13185 master_runner.cc:264] No leader master found from master c336a69c59f14d7da636e9a1b1fff164
I20260806 07:52:29.482559 13185 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:52:29.506328 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 12748
I20260806 07:52:29.526942 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35563
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--webserver_port=44533
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:29.527310 13273 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:29.499080 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:33969 (local address 127.20.202.254:40139)
0806 07:52:29.499423 (+ 343us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:29.499430 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:29.506123 (+ 6693us) server_negotiation.cc:408] Connection header received
0806 07:52:29.506537 (+ 414us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:29.506544 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:29.506629 (+ 85us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:29.506707 (+ 78us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0806 07:52:29.525893 (+ 19186us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:33969: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":188}
I20260806 07:52:29.713116 13174 heartbeater.cc:499] Master 127.20.202.254:40139 was elected leader, sending a full tablet report...
I20260806 07:52:29.724900 13218 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "0d73f52291d14582b9e0116d578a7603" instance_seqno: 1786002748655716) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:55375; Asking this server to re-register.
I20260806 07:52:29.726075 13174 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:29.728191 13218 ts_manager.cc:194] Registered new tserver with Master: 0d73f52291d14582b9e0116d578a7603 (127.20.202.195:40493)
W20260806 07:52:29.901945 13278 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:29.902266 13278 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:29.902464 13278 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:29.911188 13278 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:29.911437 13278 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:29.923286 13278 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44533
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:29.924955 13278 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:29.926405 13278 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:29.938382 13284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:29.938520 13285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:29.939442 13278 server_base.cc:1061] running on GCE node
W20260806 07:52:29.939569 13287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:29.940294 13278 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:29.940497 13278 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:29.940634 13278 hybrid_clock.cc:648] HybridClock initialized: now 1786002749940602 us; error 0 us; skew 500 ppm
I20260806 07:52:29.941053 13278 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:29 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002749, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:29.946364 13278 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:29.955539 13278 webserver.cc:533] Webserver started at http://127.20.202.193:44533/ using document root <none> and password file <none>
I20260806 07:52:29.956707 13278 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:29.956861 13278 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:29.963284 13278 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 07:52:29.966152 13293 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:29.967551 13278 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:52:29.967815 13278 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:29.968660 13278 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:29.993005 13278 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:29.997687 13278 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:29.997980 13278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:29.999138 13278 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:30.000811 13278 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:30.000941 13278 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:30.001072 13278 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:30.001174 13278 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:30.037940 13278 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35563
I20260806 07:52:30.038044 13407 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35563 every 8 connection(s)
I20260806 07:52:30.039613 13278 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:30.045825 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 13278
Aug 06 07:52:30 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:52:30 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002749, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:30 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002749, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:30 dist-test-slave-g350 krb5kdc[12273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002749, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:30.073478 13411 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38221
I20260806 07:52:30.073945 13411 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:30.074906 13411 heartbeater.cc:507] Master 127.20.202.253:38221 requested a full tablet report, sending...
I20260806 07:52:30.075886 13409 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38497
I20260806 07:52:30.076068 13409 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:30.076574 13409 heartbeater.cc:507] Master 127.20.202.252:38497 requested a full tablet report, sending...
I20260806 07:52:30.077296 12589 ts_manager.cc:194] Re-registered known tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:35563)
I20260806 07:52:30.080101 12679 ts_manager.cc:194] Re-registered known tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:35563)
I20260806 07:52:30.085059 13410 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:30.085249 13410 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:30.085875 13410 heartbeater.cc:507] Master 127.20.202.254:40139 requested a full tablet report, sending...
I20260806 07:52:30.087231 13218 ts_manager.cc:194] Registered new tserver with Master: df2fd2c67677461798da1b95cc1399f1 (127.20.202.193:35563)
I20260806 07:52:30.104012 13032 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40139
I20260806 07:52:30.104303 13032 heartbeater.cc:499] Master 127.20.202.254:40139 was elected leader, sending a full tablet report...
I20260806 07:52:30.105399 13218 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "15b7eb849fd54f218b9954353394c000" instance_seqno: 1786002748027376) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:59571; Asking this server to re-register.
I20260806 07:52:30.106256 13032 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:30.107594 13218 ts_manager.cc:194] Registered new tserver with Master: 15b7eb849fd54f218b9954353394c000 (127.20.202.194:44657)
I20260806 07:52:30.109776 13420 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c336a69c59f14d7da636e9a1b1fff164)
I20260806 07:52:30.110014 13420 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:30.112185 13420 leader_election.cc:290] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c336a69c59f14d7da636e9a1b1fff164 (127.20.202.254:40139), 28b9f7ed0d5745e19c092f948a7b7d39 (127.20.202.253:38221)
I20260806 07:52:30.112994 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca65838c4315482791c4a8a1c9bee3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" is_pre_election: true
I20260806 07:52:30.125967 13228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca65838c4315482791c4a8a1c9bee3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c336a69c59f14d7da636e9a1b1fff164" is_pre_election: true
I20260806 07:52:30.126622 13228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca65838c4315482791c4a8a1c9bee3aa in term 2.
I20260806 07:52:30.127525 12665 leader_election.cc:304] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c336a69c59f14d7da636e9a1b1fff164, ca65838c4315482791c4a8a1c9bee3aa; no voters: 28b9f7ed0d5745e19c092f948a7b7d39
I20260806 07:52:30.128574 13420 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:52:30.128715 13420 raft_consensus.cc:493] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader c336a69c59f14d7da636e9a1b1fff164)
I20260806 07:52:30.128827 13420 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:52:30.132375 13420 raft_consensus.cc:515] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:30.133905 13228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca65838c4315482791c4a8a1c9bee3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c336a69c59f14d7da636e9a1b1fff164"
I20260806 07:52:30.134147 13228 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:52:30.141316 13228 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca65838c4315482791c4a8a1c9bee3aa in term 3.
I20260806 07:52:30.142099 12665 leader_election.cc:304] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c336a69c59f14d7da636e9a1b1fff164, ca65838c4315482791c4a8a1c9bee3aa; no voters:
I20260806 07:52:30.146631 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca65838c4315482791c4a8a1c9bee3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28b9f7ed0d5745e19c092f948a7b7d39"
I20260806 07:52:30.147859 13420 leader_election.cc:290] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [CANDIDATE]: Term 3 election: Requested vote from peers c336a69c59f14d7da636e9a1b1fff164 (127.20.202.254:40139), 28b9f7ed0d5745e19c092f948a7b7d39 (127.20.202.253:38221)
I20260806 07:52:30.148417 13420 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:52:30.148960 13420 raft_consensus.cc:697] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [term 3 LEADER]: Becoming Leader. State: Replica: ca65838c4315482791c4a8a1c9bee3aa, State: Running, Role: LEADER
I20260806 07:52:30.150238 13420 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:30.152897 13422 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca65838c4315482791c4a8a1c9bee3aa. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:30.153191 13422 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:30.154312 13424 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:52:30.156699 13424 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:52:30.157963 13424 catalog_manager.cc:1295] Loaded cluster ID: a9c273fc19eb4e92bd80094a1a62dd90
I20260806 07:52:30.158142 13424 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:52:30.160056 13424 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:52:30.161319 13424 catalog_manager.cc:6081] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa: Loaded TSK: 0
I20260806 07:52:30.162669 13424 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:52:30.207100 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13278
W20260806 07:52:30.208384 13428 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:30.204585 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.20.202.193:35563)
0806 07:52:30.205381 (+ 796us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:30.205389 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:30.205421 (+ 32us) server_negotiation.cc:408] Connection header received
0806 07:52:30.205492 (+ 71us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:30.205499 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:30.205801 (+ 302us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:30.205975 (+ 174us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:30.206974 (+ 999us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49726: BlockingRecv error: recv got EOF from 127.0.0.1:49726 (error 108)
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":457,"threads_started":1}
W20260806 07:52:30.209789 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13278: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:35563: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:52:30.209833 13428 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:30.208039 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.20.202.193:35563)
0806 07:52:30.208743 (+ 704us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:30.208748 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:30.208763 (+ 15us) server_negotiation.cc:408] Connection header received
0806 07:52:30.208805 (+ 42us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:30.208808 (+ 3us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:30.208860 (+ 52us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:30.208940 (+ 80us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:30.209563 (+ 623us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49738: BlockingRecv error: recv got EOF from 127.0.0.1:49738 (error 108)
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":501,"threads_started":1}
I20260806 07:52:30.209895 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 13278
I20260806 07:52:30.230438 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35563
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--webserver_port=44533
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:30.340353 13431 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c336a69c59f14d7da636e9a1b1fff164)
I20260806 07:52:30.340583 13431 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } }
I20260806 07:52:30.342427 13431 leader_election.cc:290] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c336a69c59f14d7da636e9a1b1fff164 (127.20.202.254:40139), ca65838c4315482791c4a8a1c9bee3aa (127.20.202.252:38497)
I20260806 07:52:30.349994 12688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ca65838c4315482791c4a8a1c9bee3aa" is_pre_election: true
I20260806 07:52:30.351085 13228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c336a69c59f14d7da636e9a1b1fff164" is_pre_election: true
I20260806 07:52:30.351473 13228 raft_consensus.cc:2393] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28b9f7ed0d5745e19c092f948a7b7d39 in current term 3: Already voted for candidate ca65838c4315482791c4a8a1c9bee3aa in this term.
I20260806 07:52:30.352306 12575 leader_election.cc:304] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28b9f7ed0d5745e19c092f948a7b7d39; no voters: c336a69c59f14d7da636e9a1b1fff164, ca65838c4315482791c4a8a1c9bee3aa
I20260806 07:52:30.352752 13431 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:52:30.356036 13431 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260806 07:52:30.603020 13430 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:30.603441 13430 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:30.603621 13430 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:52:30.603710 13430 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:30.612619 13430 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:30.612883 13430 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:30.623839 13430 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44533
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:30.625698 13430 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:30.627070 13430 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:30.637159 13439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:30.637055 13438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:30.640770 13441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:30.640770 13430 server_base.cc:1061] running on GCE node
I20260806 07:52:30.641518 13430 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:30.641732 13430 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:30.641868 13430 hybrid_clock.cc:648] HybridClock initialized: now 1786002750641823 us; error 0 us; skew 500 ppm
I20260806 07:52:30.642189 13430 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:30 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:30.646122 13430 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:30.647879 13430 webserver.cc:533] Webserver started at http://127.20.202.193:44533/ using document root <none> and password file <none>
I20260806 07:52:30.648685 13430 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:30.648801 13430 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:30.654794 13430 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:52:30.657953 13447 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:30.659816 13430 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:52:30.660012 13430 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:30.660828 13430 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:30.662679 13228 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [term 3 FOLLOWER]: Refusing update from remote peer ca65838c4315482791c4a8a1c9bee3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:52:30.663766 13448 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:52:30.667627 13449 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca65838c4315482791c4a8a1c9bee3aa. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:30.667930 13449 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:30.671869 13449 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:30.672106 13449 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336a69c59f14d7da636e9a1b1fff164 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:30.673429 12599 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [term 3 FOLLOWER]: Refusing update from remote peer ca65838c4315482791c4a8a1c9bee3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:52:30.684489 13430 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:30.685753 13458 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:30.686101 13458 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:30.686982 13454 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:52:30.691413 13430 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:30.691717 13430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:30.692863 13430 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:30.694717 13466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:30.694795 13431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca65838c4315482791c4a8a1c9bee3aa. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:30.695113 13466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:30.695151 13431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b9f7ed0d5745e19c092f948a7b7d39 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:30.695398 13430 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:30.695500 13430 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:30.695662 13430 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:30.695745 13430 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:52:30.696151 13430 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:52:30.696276 13430 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:52:30.696377 13430 fault_injection.cc:59] injecting fault for kudu (pid 13430): FLAGS_fault_before_start (process will exit)
I20260806 07:52:30.721318 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:35563
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--webserver_port=44533
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:31.081571 13468 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:31.081986 13468 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:31.082161 13468 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:52:31.085703 13460 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ca65838c4315482791c4a8a1c9bee3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c336a69c59f14d7da636e9a1b1fff164" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40139 } } peers { permanent_uuid: "28b9f7ed0d5745e19c092f948a7b7d39" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38221 } } peers { permanent_uuid: "ca65838c4315482791c4a8a1c9bee3aa" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38497 } } }
I20260806 07:52:31.085987 13460 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca65838c4315482791c4a8a1c9bee3aa [sys.catalog]: This master's current role is: LEADER
W20260806 07:52:31.091421 13468 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:31.091671 13468 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:31.103470 13468 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:35563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44533
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40139,127.20.202.253:38221,127.20.202.252:38497
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:31.105358 13468 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:31.106771 13468 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:31.118131 13474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:31.118145 13473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:31.118863 13476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:31.120548 13468 server_base.cc:1061] running on GCE node
I20260806 07:52:31.121310 13468 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:52:31.121541 13468 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:52:31.121668 13468 hybrid_clock.cc:648] HybridClock initialized: now 1786002751121646 us; error 0 us; skew 500 ppm
I20260806 07:52:31.122038 13468 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:31 dist-test-slave-g350 krb5kdc[12273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002751, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:31.126425 13468 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:31.127985 13468 webserver.cc:533] Webserver started at http://127.20.202.193:44533/ using document root <none> and password file <none>
I20260806 07:52:31.128690 13468 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:31.128783 13468 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:31.134263 13468 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:52:31.136940 13482 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:31.138506 13468 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:31.138679 13468 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "df2fd2c67677461798da1b95cc1399f1"
format_stamp: "Formatted at 2026-08-06 07:52:08 on dist-test-slave-g350"
I20260806 07:52:31.139426 13468 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:31.161358 13468 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:31.165819 13468 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:31.166172 13468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:31.167223 13468 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:31.168919 13468 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:31.169010 13468 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:31.169185 13468 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:31.169276 13468 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:31.204645 13468 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35563
I20260806 07:52:31.204736 13596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35563 every 8 connection(s)
I20260806 07:52:31.206357 13468 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:31.212818 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 13468
I20260806 07:52:31.213143 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13468
W20260806 07:52:31.219417 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13468: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:35563: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:31.219648 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 13468
W20260806 07:52:31.234705 13606 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:31.212922 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:37889 (local address 127.20.202.254:40139)
0806 07:52:31.213916 (+ 994us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:31.213924 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:31.214655 (+ 731us) server_negotiation.cc:408] Connection header received
0806 07:52:31.214742 (+ 87us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:31.214751 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:31.214851 (+ 100us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:31.214975 (+ 124us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:31.234276 (+ 19301us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:37889: BlockingRecv error: recv got EOF from 127.20.202.193:37889 (error 108)
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":737,"threads_started":1}
W20260806 07:52:31.235169 13601 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:31.209764 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:43573 (local address 127.20.202.253:38221)
0806 07:52:31.210314 (+ 550us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:31.210320 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:31.211735 (+ 1415us) server_negotiation.cc:408] Connection header received
0806 07:52:31.214704 (+ 2969us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:31.214713 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:31.214801 (+ 88us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:31.214946 (+ 145us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:31.218658 (+ 3712us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:52:31.219951 (+ 1293us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:52:31.234288 (+ 14337us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:43573: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":249,"threads_started":1}
W20260806 07:52:31.235193 13602 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:31.209902 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:55261 (local address 127.20.202.252:38497)
0806 07:52:31.210645 (+ 743us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:31.210651 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:31.211787 (+ 1136us) server_negotiation.cc:408] Connection header received
0806 07:52:31.215180 (+ 3393us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:31.215187 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:31.215268 (+ 81us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:31.215386 (+ 118us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:31.217388 (+ 2002us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:52:31.219185 (+ 1797us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:52:31.234227 (+ 15042us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:55261: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":931,"threads_started":1}
I20260806 07:52:31.235688 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12899
W20260806 07:52:31.238898 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12899: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:44657: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:31.239078 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 12899
I20260806 07:52:31.256721 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13045
W20260806 07:52:31.259799 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13045: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:40493: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:31.259964 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 13045
I20260806 07:52:31.277510 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13185
W20260806 07:52:31.279866 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13185: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:40139: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:52:31.280030 13606 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:31.278548 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.20.202.254:40139)
0806 07:52:31.278745 (+ 197us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:31.278749 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:31.278768 (+ 19us) server_negotiation.cc:408] Connection header received
0806 07:52:31.278941 (+ 173us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:31.278946 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:31.279012 (+ 66us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:31.279123 (+ 111us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:31.279767 (+ 644us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46726: BlockingRecv error: recv got EOF from 127.0.0.1:46726 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 07:52:31.280054 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 13185
I20260806 07:52:31.302759 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12558
I20260806 07:52:31.304049 12619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:31.418314 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 12558
I20260806 07:52:31.441174 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 12648
I20260806 07:52:31.442571 12709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:31.553328 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 12648
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestBasicOperation/9 (47224 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/10
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:52:32 dist-test-slave-g350 krb5kdc[13620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:52:32 dist-test-slave-g350 krb5kdc[13620](info): set up 2 sockets
Aug 06 07:52:32 dist-test-slave-g350 krb5kdc[13620](info): commencing operation
krb5kdc: starting...
W20260806 07:52:34.085510 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.111s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:52:34 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:52:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:52:34Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:52:42: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:52:44 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002764, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:52:50 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002770, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:52:50 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002770, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:50.969182 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:51.313526 13920 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:51.313962 13920 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:51.314126 13920 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:51.322486 13920 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:51.322629 13920 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:51.322751 13920 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:51.322829 13920 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:51.333979 13920 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:51.335822 13920 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:51.337245 13920 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:51.345794 13927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:51.345743 13926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:51.346876 13929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:51.347328 13920 server_base.cc:1061] running on GCE node
I20260806 07:52:51.348155 13920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:51.349712 13920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:51.351007 13920 hybrid_clock.cc:648] HybridClock initialized: now 1786002771350897 us; error 57 us; skew 500 ppm
I20260806 07:52:51.351404 13920 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:51 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:51.354743 13920 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:52:51.356528 13920 webserver.cc:533] Webserver started at http://127.20.202.254:37213/ using document root <none> and password file <none>
I20260806 07:52:51.357250 13920 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:51.357344 13920 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:51.357725 13920 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:51.360172 13920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "82c11602542a4f7c97ecf42cb4a97678"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:51.360924 13920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "82c11602542a4f7c97ecf42cb4a97678"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:51.366701 13920 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 07:52:51.370286 13936 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:51.371688 13920 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:52:51.371836 13920 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "82c11602542a4f7c97ecf42cb4a97678"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:51.372011 13920 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:51.399466 13920 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:51.406975 13920 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:51.407244 13920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:51.429193 13920 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43837
I20260806 07:52:51.429232 13987 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43837 every 8 connection(s)
I20260806 07:52:51.431054 13920 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:52:51.435436 13988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:52:51.437366 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 13920
I20260806 07:52:51.437646 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:52:51.443938 13988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:51 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
I20260806 07:52:51.469054 13988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
W20260806 07:52:51.470357 13940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111)
W20260806 07:52:51.471925 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39499: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:51.521294 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } attempt: 1
W20260806 07:52:51.523554 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39499: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:51.560745 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39499
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:51.591837 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } attempt: 2
W20260806 07:52:51.594113 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39499: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111)
I20260806 07:52:51.685416 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } attempt: 3
W20260806 07:52:51.687775 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39499: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111)
I20260806 07:52:51.831115 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } attempt: 4
W20260806 07:52:51.833750 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:39499: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111)
W20260806 07:52:51.909731 13997 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:51.910048 13997 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:51.910197 13997 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:51.918480 13997 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:51.918582 13997 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:51.918671 13997 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:51.918742 13997 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:51.930055 13997 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:51.931727 13997 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:51.933168 13997 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:51.944736 14002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:51.945318 14003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:51.947113 14005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:51.948128 13997 server_base.cc:1061] running on GCE node
I20260806 07:52:51.948954 13997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:51.950614 13997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:51.951864 13997 hybrid_clock.cc:648] HybridClock initialized: now 1786002771951824 us; error 32 us; skew 500 ppm
I20260806 07:52:51.952235 13997 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:51 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:51.955470 13997 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:52:51.957100 13997 webserver.cc:533] Webserver started at http://127.20.202.253:39389/ using document root <none> and password file <none>
I20260806 07:52:51.957906 13997 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:51.958047 13997 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:51.958407 13997 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:51.960807 13997 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "d633d87bb50447589745984b4100056c"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:51.961637 13997 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "d633d87bb50447589745984b4100056c"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:51.967186 13997 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:52:51.970645 14012 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:51.971925 13997 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:52:51.972110 13997 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "d633d87bb50447589745984b4100056c"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:51.972309 13997 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:52.003111 13997 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:52.007252 13997 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:52.007666 13997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:52.028650 13997 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:39499
I20260806 07:52:52.028753 14065 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:39499 every 8 connection(s)
I20260806 07:52:52.030283 13997 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:52:52.034904 14066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:52:52.039153 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 13997
I20260806 07:52:52.039340 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:52:52.040970 14066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:52.061828 14066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:52.079288 14066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:52:52.080653 14016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
W20260806 07:52:52.082677 14066 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:52:52.132176 14066 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 1
I20260806 07:52:52.133054 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } attempt: 5
W20260806 07:52:52.134728 14066 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:52.147516 13988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:52:52.149471 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:52.160821 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38561
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:52.200702 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 1
W20260806 07:52:52.202761 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
I20260806 07:52:52.202965 14066 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 2
W20260806 07:52:52.204717 14066 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
I20260806 07:52:52.271054 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 2
W20260806 07:52:52.273195 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
I20260806 07:52:52.295970 14066 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 3
W20260806 07:52:52.298235 14066 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
I20260806 07:52:52.379485 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 3
W20260806 07:52:52.381575 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
I20260806 07:52:52.441470 14066 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 4
W20260806 07:52:52.443619 14066 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
W20260806 07:52:52.506294 14076 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:52.506621 14076 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:52.506806 14076 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:52.515347 14076 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:52.515477 14076 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:52.515621 14076 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:52.515709 14076 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:52.526517 14076 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:52.528273 14076 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:52.529734 14076 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:52.538606 14082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:52.538779 14081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:52.538595 14084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:52.539700 14076 server_base.cc:1061] running on GCE node
I20260806 07:52:52.540475 14076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:52.542111 14076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:52.543349 14076 hybrid_clock.cc:648] HybridClock initialized: now 1786002772543286 us; error 56 us; skew 500 ppm
I20260806 07:52:52.543725 14076 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002772, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:52.547045 14076 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:52:52.548659 14076 webserver.cc:533] Webserver started at http://127.20.202.252:40235/ using document root <none> and password file <none>
I20260806 07:52:52.549340 14076 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:52.549443 14076 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:52.549819 14076 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:52.552163 14076 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b"
format_stamp: "Formatted at 2026-08-06 07:52:52 on dist-test-slave-g350"
I20260806 07:52:52.552897 14076 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b"
format_stamp: "Formatted at 2026-08-06 07:52:52 on dist-test-slave-g350"
I20260806 07:52:52.558284 14076 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 07:52:52.558854 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 4
W20260806 07:52:52.560905 13988 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111)
I20260806 07:52:52.562017 14091 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:52.563491 14076 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:52:52.563692 14076 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b"
format_stamp: "Formatted at 2026-08-06 07:52:52 on dist-test-slave-g350"
I20260806 07:52:52.563923 14076 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:52.586313 14076 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:52.590338 14076 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:52.590631 14076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:52.612147 14076 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:38561
I20260806 07:52:52.612186 14142 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:38561 every 8 connection(s)
I20260806 07:52:52.613788 14076 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:52:52.618047 14143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:52:52.619427 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14076
I20260806 07:52:52.619676 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:52:52.625169 14143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002772, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:52.646832 14143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002772, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:52.661788 14143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002772, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:52.682296 14143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Bootstrap starting.
I20260806 07:52:52.685585 14143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Neither blocks nor log segments found. Creating new log.
I20260806 07:52:52.686705 14143 log.cc:826] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:52.689251 14143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: No bootstrap required, opened a new log
I20260806 07:52:52.694815 14143 raft_consensus.cc:359] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.695149 14143 raft_consensus.cc:385] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:52:52.695287 14143 raft_consensus.cc:740] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e7ce51c5b2478288bc91b7fd7f2d1b, State: Initialized, Role: FOLLOWER
I20260806 07:52:52.695900 14143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.696769 14153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.697055 14153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:52.697903 14143 sys_catalog.cc:565] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:52:52.711692 14164 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:52:52.711828 14164 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:52:52.705268 14076 master_runner.cc:187] Error getting master registration for 127.20.202.254:43837: OK, instance_id { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" instance_seqno: 1786002771423393 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:52:52.712564 14076 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:52:52.741640 14153 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:52:52.741936 14153 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.742861 14066 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 5
I20260806 07:52:52.744078 14153 leader_election.cc:290] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837), d633d87bb50447589745984b4100056c (127.20.202.253:39499)
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:52.764770 14066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Bootstrap starting.
I20260806 07:52:52.769181 14066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Neither blocks nor log segments found. Creating new log.
I20260806 07:52:52.770685 14066 log.cc:826] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:52.774546 14066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: No bootstrap required, opened a new log
I20260806 07:52:52.782645 14066 raft_consensus.cc:359] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.783021 14066 raft_consensus.cc:385] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:52:52.783160 14066 raft_consensus.cc:740] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d633d87bb50447589745984b4100056c, State: Initialized, Role: FOLLOWER
I20260806 07:52:52.783900 14066 consensus_queue.cc:260] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.784945 14167 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.785265 14167 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:52.786365 14066 sys_catalog.cc:565] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:52.786412 14039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d633d87bb50447589745984b4100056c" is_pre_election: true
I20260806 07:52:52.786973 14039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64e7ce51c5b2478288bc91b7fd7f2d1b in term 0.
I20260806 07:52:52.788038 14095 leader_election.cc:304] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64e7ce51c5b2478288bc91b7fd7f2d1b, d633d87bb50447589745984b4100056c; no voters:
I20260806 07:52:52.788568 14153 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:52:52.788743 14153 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:52:52.788866 14153 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:52:52.792891 14153 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.795280 14153 leader_election.cc:290] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 1 election: Requested vote from peers 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837), d633d87bb50447589745984b4100056c (127.20.202.253:39499)
I20260806 07:52:52.795254 14039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d633d87bb50447589745984b4100056c"
I20260806 07:52:52.795627 14039 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:52:52.799561 14039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64e7ce51c5b2478288bc91b7fd7f2d1b in term 1.
I20260806 07:52:52.800186 14095 leader_election.cc:304] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64e7ce51c5b2478288bc91b7fd7f2d1b, d633d87bb50447589745984b4100056c; no voters:
I20260806 07:52:52.800530 14153 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:52:52.801141 14153 raft_consensus.cc:697] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 1 LEADER]: Becoming Leader. State: Replica: 64e7ce51c5b2478288bc91b7fd7f2d1b, State: Running, Role: LEADER
I20260806 07:52:52.801766 14153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.805689 14173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.806384 14173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:52.817179 14177 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:52:52.819836 14177 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:52:52.829703 14039 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 1 FOLLOWER]: Refusing update from remote peer 64e7ce51c5b2478288bc91b7fd7f2d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:52:52.830777 14173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:52:52.838131 13988 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } attempt: 5
W20260806 07:52:52.842832 14097 tablet.cc:2406] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 07:52:52 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002771, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:52.843628 13997 master_runner.cc:187] Error getting master registration for 127.20.202.254:43837: OK, instance_id { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" instance_seqno: 1786002771423393 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:52:52.850128 13997 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:52:52.859736 14167 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.860052 14167 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:52.864219 14180 mvcc.cc:204] Tried to move back new op lower bound from 7315467357497794560 to 7315467357401239552. Current Snapshot: MvccSnapshot[applied={T|T < 7315467357497794560}]
W20260806 07:52:52.865015 14183 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:52:52.865139 14183 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:52:52.873203 13988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap starting.
I20260806 07:52:52.876962 14173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.877259 14173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:52.884614 13988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Neither blocks nor log segments found. Creating new log.
I20260806 07:52:52.884042 14184 mvcc.cc:204] Tried to move back new op lower bound from 7315467357497794560 to 7315467357401239552. Current Snapshot: MvccSnapshot[applied={T|T < 7315467357497794560}]
I20260806 07:52:52.887760 13988 log.cc:826] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:52.888628 14167 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.888895 14167 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:52.900177 13988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: No bootstrap required, opened a new log
I20260806 07:52:52.902693 14177 catalog_manager.cc:1383] Generated new cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:52:52.902817 14177 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:52:52.919643 13988 raft_consensus.cc:359] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.920351 13988 raft_consensus.cc:385] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:52:52.920626 13988 raft_consensus.cc:740] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c11602542a4f7c97ecf42cb4a97678, State: Initialized, Role: FOLLOWER
I20260806 07:52:52.922113 13988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:52.938238 14190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.938591 14190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:52.942440 13963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c11602542a4f7c97ecf42cb4a97678" is_pre_election: true
I20260806 07:52:52.942830 13961 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:52:52.944340 13962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c11602542a4f7c97ecf42cb4a97678"
I20260806 07:52:52.945422 13988 sys_catalog.cc:565] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:52.946740 13961 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Refusing update from remote peer 64e7ce51c5b2478288bc91b7fd7f2d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:52:52.948040 14173 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:52:52.963810 14177 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:52:52.965631 14177 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:52:52.989138 14173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.989387 14173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:52.989253 14190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:52.989519 14190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:52.990531 14177 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Generated new TSK 0
I20260806 07:52:52.991740 14177 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:52:53.003551 14195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:53.003919 14195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:53.028803 14207 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:52:53.033906 14207 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: loading cluster ID for follower catalog manager: success
I20260806 07:52:53.038543 14207 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: acquiring CA information for follower catalog manager: success
I20260806 07:52:53.044570 14207 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
W20260806 07:52:53.114602 21291 mini_kdc.cc:268] Time spent creating service keytab for kudu/127.20.202.193: real 0.100s user 0.003s sys 0.020s
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:53.193159 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:53.674027 14213 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:53.674376 14213 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:53.674520 14213 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:53.683441 14213 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:53.683686 14213 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:53.694396 14213 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:53.696033 14213 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:53.697434 14213 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:53.708045 14219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:53.708043 14218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:53.708348 14221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:53.708847 14213 server_base.cc:1061] running on GCE node
I20260806 07:52:53.709498 14213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:53.710480 14213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:53.711696 14213 hybrid_clock.cc:648] HybridClock initialized: now 1786002773711648 us; error 32 us; skew 500 ppm
I20260806 07:52:53.712095 14213 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:53 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002773, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:53.715556 14213 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:53.717310 14213 webserver.cc:533] Webserver started at http://127.20.202.193:36771/ using document root <none> and password file <none>
I20260806 07:52:53.718153 14213 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:53.718299 14213 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:53.718674 14213 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:53.721125 14213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:52:53.722007 14213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:52:53.727526 14213 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:52:53.731132 14228 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:53.732704 14213 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:52:53.732915 14213 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:52:53.733168 14213 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:53.766168 14213 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:53.770844 14213 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:53.771343 14213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:53.772403 14213 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:53.774166 14213 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:53.774273 14213 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:53.774422 14213 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:53.774495 14213 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:53.810636 14213 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36105
I20260806 07:52:53.810696 14342 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36105 every 8 connection(s)
I20260806 07:52:53.812146 14213 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:53.822589 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14213
I20260806 07:52:53.822839 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:52:53 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002773, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:53 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002773, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:53 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002773, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:53.842509 14345 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:52:53.842573 14343 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:52:53.843014 14345 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:53.843010 14343 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:53.843848 14344 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:52:53.844022 14344 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:53.844362 14343 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:52:53.844403 14345 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:52:53.844570 14344 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:52:53.847286 13952 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:36105)
I20260806 07:52:53.847296 14108 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:36105)
I20260806 07:52:53.847488 14029 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:36105)
I20260806 07:52:53.849171 14108 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:52685
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.194@KRBTEST.COM" created.
I20260806 07:52:53.867125 14183 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:52:53.867208 14183 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: loading cluster ID for follower catalog manager: success
I20260806 07:52:53.870067 14183 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: acquiring CA information for follower catalog manager: success
I20260806 07:52:53.871973 14183 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:53.961467 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:54.327450 14357 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:54.327780 14357 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:54.327971 14357 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:54.336632 14357 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:54.336892 14357 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:52:54.348788 14357 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:54.350658 14357 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:54.352085 14357 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:54.362514 14363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:54.362541 14362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:54.362870 14365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:54.363471 14357 server_base.cc:1061] running on GCE node
I20260806 07:52:54.364080 14357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:54.365041 14357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:54.366279 14357 hybrid_clock.cc:648] HybridClock initialized: now 1786002774366221 us; error 43 us; skew 500 ppm
I20260806 07:52:54.366632 14357 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:54 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002774, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:54.370121 14357 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:52:54.371760 14357 webserver.cc:533] Webserver started at http://127.20.202.194:38773/ using document root <none> and password file <none>
I20260806 07:52:54.372524 14357 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:54.372660 14357 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:54.373001 14357 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:54.375524 14357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "5b98d06cc9a0440daf3add1863ecb4d5"
format_stamp: "Formatted at 2026-08-06 07:52:54 on dist-test-slave-g350"
I20260806 07:52:54.376327 14357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "5b98d06cc9a0440daf3add1863ecb4d5"
format_stamp: "Formatted at 2026-08-06 07:52:54 on dist-test-slave-g350"
I20260806 07:52:54.382207 14357 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 07:52:54.385733 14372 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:54.387266 14357 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:52:54.387475 14357 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "5b98d06cc9a0440daf3add1863ecb4d5"
format_stamp: "Formatted at 2026-08-06 07:52:54 on dist-test-slave-g350"
I20260806 07:52:54.387724 14357 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:54.412369 14357 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:54.417184 14357 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:54.417537 14357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:54.418677 14357 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:54.420432 14357 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:54.420544 14357 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:54.420701 14357 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:54.420786 14357 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:54.460031 14357 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:35779
I20260806 07:52:54.460069 14485 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:35779 every 8 connection(s)
I20260806 07:52:54.461608 14357 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:52:54.471379 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14357
I20260806 07:52:54.471619 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:52:54 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002774, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:54 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002774, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:54 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002774, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:52:54.500124 14488 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:52:54.500572 14488 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:54.501787 14488 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:52:54.503141 14486 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:52:54.503281 14486 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:54.503918 14486 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:52:54.504215 14029 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35779)
I20260806 07:52:54.505079 14108 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35779)
I20260806 07:52:54.506351 14108 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:54119
I20260806 07:52:54.506697 14487 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:52:54.506832 14487 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:54.507398 14487 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:52:54.508656 13952 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35779)
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:52:54.609308 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:54.851691 14343 heartbeater.cc:499] Master 127.20.202.252:38561 was elected leader, sending a full tablet report...
W20260806 07:52:54.972465 14500 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:54.972777 14500 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:54.972867 14500 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:54.981338 14500 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:54.981515 14500 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:52:54.992038 14500 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:54.993619 14500 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:54.994998 14500 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:55.004913 14505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:55.005581 14506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:55.006356 14508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:55.007054 14500 server_base.cc:1061] running on GCE node
I20260806 07:52:55.007644 14500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:55.008677 14500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:55.009945 14500 hybrid_clock.cc:648] HybridClock initialized: now 1786002775009881 us; error 48 us; skew 500 ppm
I20260806 07:52:55.010293 14500 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:55 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:55.013888 14500 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:52:55.015658 14500 webserver.cc:533] Webserver started at http://127.20.202.195:42121/ using document root <none> and password file <none>
I20260806 07:52:55.016505 14500 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:55.016717 14500 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:55.017108 14500 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:52:55.019690 14500 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "dadf93bef9884c7d98465809491d2626"
format_stamp: "Formatted at 2026-08-06 07:52:55 on dist-test-slave-g350"
I20260806 07:52:55.020594 14500 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "dadf93bef9884c7d98465809491d2626"
format_stamp: "Formatted at 2026-08-06 07:52:55 on dist-test-slave-g350"
I20260806 07:52:55.026016 14500 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:52:55.029706 14515 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:55.031186 14500 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:52:55.031392 14500 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "dadf93bef9884c7d98465809491d2626"
format_stamp: "Formatted at 2026-08-06 07:52:55 on dist-test-slave-g350"
I20260806 07:52:55.031647 14500 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:55.051664 14500 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:55.055632 14500 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:55.055954 14500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:55.057001 14500 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:55.058782 14500 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:55.058897 14500 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:55.059051 14500 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:55.059124 14500 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:55.096182 14500 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:37899
I20260806 07:52:55.096269 14628 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:37899 every 8 connection(s)
I20260806 07:52:55.097713 14500 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:52:55.099903 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14500
I20260806 07:52:55.100136 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:52:55 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:52:55 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:55 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:52:55.129374 14629 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:52:55.129514 14631 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:52:55.129562 14630 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:52:55.129884 14630 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:55.129884 14629 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:55.129889 14631 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:55.130988 14631 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:52:55.131049 14629 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:52:55.131021 14630 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:52:55.132752 14108 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:37899)
I20260806 07:52:55.133041 13952 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:37899)
I20260806 07:52:55.133046 14029 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:37899)
I20260806 07:52:55.133929 14108 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:49601
I20260806 07:52:55.137763 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:52:55.145295 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:52:55.145484 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13920
I20260806 07:52:55.146435 13983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:55.257431 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 13920
I20260806 07:52:55.281486 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--webserver_interface=127.20.202.254
--webserver_port=37213
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:55.465190 14095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:43837: connect: Connection refused (error 111)
W20260806 07:52:55.468235 14095 consensus_peers.cc:597] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b -> Peer 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837): Couldn't send request to peer 82c11602542a4f7c97ecf42cb4a97678. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:43837: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:52:55.509405 14486 heartbeater.cc:499] Master 127.20.202.252:38561 was elected leader, sending a full tablet report...
W20260806 07:52:55.510993 14487 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:43837 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:43837: connect: Connection refused (error 111)
W20260806 07:52:55.646834 14640 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:55.647178 14640 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:55.647274 14640 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:55.656389 14640 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:52:55.656498 14640 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:55.656555 14640 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:52:55.656595 14640 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:52:55.667513 14640 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=37213
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:55.669081 14640 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:55.670650 14640 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:55.679229 14648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:55.679440 14649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:55.679922 14651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:55.681304 14640 server_base.cc:1061] running on GCE node
I20260806 07:52:55.682197 14640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:55.683895 14640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:55.685137 14640 hybrid_clock.cc:648] HybridClock initialized: now 1786002775685092 us; error 29 us; skew 500 ppm
I20260806 07:52:55.685572 14640 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:55 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:55.689258 14640 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:52:55.690995 14640 webserver.cc:533] Webserver started at http://127.20.202.254:37213/ using document root <none> and password file <none>
I20260806 07:52:55.691735 14640 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:55.691893 14640 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:55.697367 14640 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:55.700488 14658 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:55.702127 14640 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:55.702306 14640 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "82c11602542a4f7c97ecf42cb4a97678"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:52:55.703029 14640 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:55.722944 14640 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:55.727231 14640 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:55.727525 14640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:55.749908 14640 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43837
I20260806 07:52:55.749915 14709 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43837 every 8 connection(s)
I20260806 07:52:55.752076 14640 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:52:55.759688 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14640
I20260806 07:52:55.760056 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14213
I20260806 07:52:55.760047 14710 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:52:55.764945 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap starting.
Aug 06 07:52:55 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:52:55.783768 14338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:55.789549 14710 log.cc:826] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Log is configured to *not* fsync() on all Append() calls
I20260806 07:52:55.801265 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:52:55.801991 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap complete.
I20260806 07:52:55.812572 14710 raft_consensus.cc:359] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:55.813553 14710 raft_consensus.cc:740] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c11602542a4f7c97ecf42cb4a97678, State: Initialized, Role: FOLLOWER
I20260806 07:52:55.814500 14710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:52:55.815418 14716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:55.815716 14716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:55.817095 14710 sys_catalog.cc:565] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:52:55.828261 14727 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:52:55.828377 14727 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: loading cluster ID for follower catalog manager: success
I20260806 07:52:55.832753 14727 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: acquiring CA information for follower catalog manager: success
I20260806 07:52:55.835511 14727 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:52:55.924574 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14213
W20260806 07:52:55.946360 14730 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:55.923376 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:60535 (local address 127.20.202.254:43837)
0806 07:52:55.924754 (+ 1378us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:55.924763 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:55.924785 (+ 22us) server_negotiation.cc:408] Connection header received
0806 07:52:55.924865 (+ 80us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:55.924874 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:55.924966 (+ 92us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:55.925058 (+ 92us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0806 07:52:55.944851 (+ 19793us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:60535: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":1225}
I20260806 07:52:55.946487 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36105
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--webserver_port=36771
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:52:56.016952 14153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:56.017293 14153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:56.048431 14716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:56.048758 14716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:52:56.136574 14629 heartbeater.cc:499] Master 127.20.202.252:38561 was elected leader, sending a full tablet report...
I20260806 07:52:56.145833 14675 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dadf93bef9884c7d98465809491d2626" instance_seqno: 1786002775087625) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:42237; Asking this server to re-register.
I20260806 07:52:56.146710 14630 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:56.148608 14675 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:37899)
W20260806 07:52:56.352363 14734 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:56.352675 14734 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:56.352770 14734 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:56.361429 14734 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:56.361783 14734 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:56.373134 14734 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=36771
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:56.374943 14734 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:56.376322 14734 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:56.386196 14745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:56.386185 14744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:56.386245 14747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:56.388175 14734 server_base.cc:1061] running on GCE node
I20260806 07:52:56.388702 14734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:56.389703 14734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:56.390928 14734 hybrid_clock.cc:648] HybridClock initialized: now 1786002776390888 us; error 29 us; skew 500 ppm
I20260806 07:52:56.391220 14734 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:56 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:56.394784 14734 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:56.396529 14734 webserver.cc:533] Webserver started at http://127.20.202.193:36771/ using document root <none> and password file <none>
I20260806 07:52:56.397286 14734 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:56.397439 14734 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:56.403211 14734 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:52:56.406097 14754 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:56.407581 14734 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:52:56.407711 14734 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:52:56.408443 14734 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:56.424113 14734 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:56.433001 14734 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:56.433255 14734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:56.434305 14734 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:56.436007 14734 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:56.436096 14734 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:56.436173 14734 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:56.436280 14734 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:56.439934 14735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:52:56.440236 14735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:52:56.472314 14734 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36105
I20260806 07:52:56.472366 14866 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36105 every 8 connection(s)
I20260806 07:52:56.473915 14734 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:56.481544 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14734
Aug 06 07:52:56 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:52:56 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:52:56 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:52:56 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:52:56.508350 14869 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:52:56.508808 14869 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:56.509896 14869 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:52:56.512217 14029 ts_manager.cc:194] Re-registered known tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:36105)
I20260806 07:52:56.518468 14867 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:52:56.518524 14868 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:52:56.518666 14867 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:56.518697 14868 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:56.519141 14867 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:52:56.519948 14868 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:52:56.520392 14108 ts_manager.cc:194] Re-registered known tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:36105)
I20260806 07:52:56.521148 14675 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:36105)
I20260806 07:52:56.521411 14108 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:38543
I20260806 07:52:56.524626 14487 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:52:56.525779 14675 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5b98d06cc9a0440daf3add1863ecb4d5" instance_seqno: 1786002774451314) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:41443; Asking this server to re-register.
I20260806 07:52:56.526386 14487 heartbeater.cc:461] Registering TS with master...
I20260806 07:52:56.527621 14675 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35779)
I20260806 07:52:56.662118 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14734
W20260806 07:52:56.663638 14883 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:56.659494 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.20.202.193:36105)
0806 07:52:56.660324 (+ 830us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:56.660331 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:56.660360 (+ 29us) server_negotiation.cc:408] Connection header received
0806 07:52:56.660419 (+ 59us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:56.660425 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:56.660649 (+ 224us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:56.660846 (+ 197us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:56.661939 (+ 1093us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57430: BlockingRecv error: recv got EOF from 127.0.0.1:57430 (error 108)
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":429,"threads_started":1}
W20260806 07:52:56.664706 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14734: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:36105: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:56.664844 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14734
I20260806 07:52:56.686985 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36105
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--webserver_port=36771
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:57.069669 14885 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:57.069993 14885 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:57.070187 14885 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:52:57.070257 14885 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:57.078743 14885 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:57.079072 14885 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:57.090050 14885 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=36771
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:57.091699 14885 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:57.093230 14885 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:57.103408 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:57.103438 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:57.104152 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:57.104804 14885 server_base.cc:1061] running on GCE node
I20260806 07:52:57.105405 14885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:57.106383 14885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:57.107615 14885 hybrid_clock.cc:648] HybridClock initialized: now 1786002777107549 us; error 34 us; skew 500 ppm
I20260806 07:52:57.107939 14885 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:57 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002777, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:57.111687 14885 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:57.113238 14885 webserver.cc:533] Webserver started at http://127.20.202.193:36771/ using document root <none> and password file <none>
I20260806 07:52:57.114022 14885 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:57.114125 14885 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:57.119735 14885 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:52:57.122864 14900 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:57.124310 14885 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:52:57.124495 14885 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:52:57.125263 14885 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:57.139461 14885 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:57.144454 14885 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:57.144786 14885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:57.145850 14885 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:57.147511 14885 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:57.147590 14885 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:57.147734 14885 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:57.147807 14885 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:52:57.148229 14885 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:52:57.148293 14885 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:52:57.148394 14885 fault_injection.cc:59] injecting fault for kudu (pid 14885): FLAGS_fault_before_start (process will exit)
I20260806 07:52:57.171835 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:36105
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--webserver_port=36771
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:52:57.518873 14908 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:52:57.519259 14908 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:52:57.519430 14908 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:52:57.529179 14908 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:52:57.529425 14908 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:52:57.540884 14908 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:36105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=36771
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:52:57.542654 14908 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:52:57.544116 14908 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:52:57.554083 14914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:57.554139 14913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:52:57.555212 14916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:52:57.556605 14908 server_base.cc:1061] running on GCE node
I20260806 07:52:57.557302 14908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:52:57.558421 14908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:52:57.559674 14908 hybrid_clock.cc:648] HybridClock initialized: now 1786002777559606 us; error 53 us; skew 500 ppm
I20260806 07:52:57.560021 14908 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:52:57 dist-test-slave-g350 krb5kdc[13620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002777, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:52:57.563586 14908 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:52:57.565361 14908 webserver.cc:533] Webserver started at http://127.20.202.193:36771/ using document root <none> and password file <none>
I20260806 07:52:57.566200 14908 fs_manager.cc:362] Metadata directory not provided
I20260806 07:52:57.566339 14908 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:52:57.571943 14908 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:52:57.574811 14923 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:57.576543 14908 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:52:57.576761 14908 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:52:57.577765 14908 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:52:57.593649 14908 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:52:57.598208 14908 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:52:57.598621 14908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:52:57.599720 14908 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:52:57.601666 14908 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:52:57.601792 14908 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 07:52:57.601935 14908 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:52:57.602123 14908 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:52:57.639535 14908 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:36105
I20260806 07:52:57.639628 15035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:36105 every 8 connection(s)
I20260806 07:52:57.641268 14908 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:52:57.651096 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 14908
I20260806 07:52:57.651517 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14908
Aug 06 07:52:57 dist-test-slave-g350 krb5kdc[13620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002777, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
W20260806 07:52:57.660887 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14908: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:36105: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:57.661095 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14908
I20260806 07:52:57.679221 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14357
W20260806 07:52:57.679334 15043 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:57.647129 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:40109 (local address 127.20.202.254:43837)
0806 07:52:57.669390 (+ 22261us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:57.669396 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:57.669418 (+ 22us) server_negotiation.cc:408] Connection header received
0806 07:52:57.669484 (+ 66us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:57.669488 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:57.669556 (+ 68us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:57.669724 (+ 168us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:57.678920 (+ 9196us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:40109: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":22144,"thread_start_us":22027,"threads_started":1}
W20260806 07:52:57.679948 15044 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:57.647956 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:36025 (local address 127.20.202.252:38561)
0806 07:52:57.659930 (+ 11974us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:57.659936 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:57.678923 (+ 18987us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:36025: BlockingRecv error: recv got EOF from 127.20.202.193:36025 (error 108)
Metrics: {"server-negotiator.queue_time_us":11864,"thread_start_us":11762,"threads_started":1}
W20260806 07:52:57.680022 15040 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:57.644868 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:52389 (local address 127.20.202.253:39499)
0806 07:52:57.645500 (+ 632us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:57.645505 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:57.646411 (+ 906us) server_negotiation.cc:408] Connection header received
0806 07:52:57.649385 (+ 2974us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:57.649394 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:57.649508 (+ 114us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:57.649703 (+ 195us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:57.652845 (+ 3142us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:52:57.654795 (+ 1950us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:52:57.656440 (+ 1645us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:52:57.656961 (+ 521us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:52:57.679115 (+ 22154us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:52389: BlockingRecv error: recv got EOF from 127.20.202.193:52389 (error 108)
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":366,"threads_started":1}
W20260806 07:52:57.682298 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14357: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:35779: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:57.682473 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14357
I20260806 07:52:57.699841 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14500
W20260806 07:52:57.702821 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14500: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:37899: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:52:57.702978 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14500
I20260806 07:52:57.718822 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14640
W20260806 07:52:57.721196 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14640: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:43837: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:52:57.721253 15043 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:52:57.719773 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.20.202.254:43837)
0806 07:52:57.719925 (+ 152us) server_negotiation.cc:207] Beginning negotiation
0806 07:52:57.719929 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:52:57.719946 (+ 17us) server_negotiation.cc:408] Connection header received
0806 07:52:57.720080 (+ 134us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:52:57.720084 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:52:57.720164 (+ 80us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:52:57.720277 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:52:57.721026 (+ 749us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50114: BlockingRecv error: recv got EOF from 127.0.0.1:50114 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 07:52:57.721376 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14640
I20260806 07:52:57.744187 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 13997
I20260806 07:52:57.745704 14061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:57.857350 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 13997
W20260806 07:52:57.871567 14095 consensus_peers.cc:597] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b -> Peer 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837): Couldn't send request to peer 82c11602542a4f7c97ecf42cb4a97678. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:43837: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:52:57.880491 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 14076
I20260806 07:52:57.881814 14138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:52:58.001187 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 14076
Shutting down hive metastore.
2026-08-06T07:52:58Z chronyd exiting
Aug 06 07:52:58 dist-test-slave-g350 krb5kdc[15060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:52:58 dist-test-slave-g350 krb5kdc[15060](info): set up 2 sockets
Aug 06 07:52:58 dist-test-slave-g350 krb5kdc[15060](info): commencing operation
krb5kdc: starting...
W20260806 07:53:00.429069 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.013s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:53:00 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:53:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:00Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "hive/127.0.0.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:53:01: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:53:03 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002783, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:53:08 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002788, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:53:08 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002788, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:10.037814 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:10.384915 15253 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:10.385215 15253 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:10.385298 15253 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:10.393563 15253 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:10.393721 15253 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:10.393781 15253 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:10.393829 15253 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:10.404651 15253 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:10.406370 15253 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:10.407727 15253 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:10.416487 15258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:10.416510 15259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:10.416661 15261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:10.418448 15253 server_base.cc:1061] running on GCE node
I20260806 07:53:10.419133 15253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:10.420495 15253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:10.421723 15253 hybrid_clock.cc:648] HybridClock initialized: now 1786002790421669 us; error 37 us; skew 500 ppm
I20260806 07:53:10.422117 15253 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:10 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002790, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:10.425791 15253 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:53:10.427424 15253 webserver.cc:533] Webserver started at http://127.20.202.254:33689/ using document root <none> and password file <none>
I20260806 07:53:10.428174 15253 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:10.428314 15253 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:10.434166 15253 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:53:10.437124 15268 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:10.438684 15253 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:10.438877 15253 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "82c11602542a4f7c97ecf42cb4a97678"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:53:10.439627 15253 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:10.462265 15253 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:10.470433 15253 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:10.470762 15253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:10.492161 15253 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43837
I20260806 07:53:10.492202 15319 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43837 every 8 connection(s)
I20260806 07:53:10.493907 15253 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:53:10.497056 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15253
I20260806 07:53:10.497346 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:53:10.503575 15320 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:10.509384 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:10.529353 15320 log.cc:826] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:10.540675 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:10.541318 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap complete.
I20260806 07:53:10.549326 15320 raft_consensus.cc:359] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:10.550216 15320 raft_consensus.cc:740] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c11602542a4f7c97ecf42cb4a97678, State: Initialized, Role: FOLLOWER
I20260806 07:53:10.551015 15320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:10.552040 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:10.552363 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:10.553292 15320 sys_catalog.cc:565] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:10.565050 15336 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:53:10.565229 15336 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: loading cluster ID for follower catalog manager: success
I20260806 07:53:10.569676 15336 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: acquiring CA information for follower catalog manager: success
I20260806 07:53:10.572374 15336 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:10.590616 15253 master_runner.cc:187] Error getting master registration for 127.20.202.253:39499: Network error: Client connection negotiation failed: client connection to 127.20.202.253:39499: connect: Connection refused (error 111),
I20260806 07:53:10.596365 15253 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:10.628758 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39499
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:10.986769 15342 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:10.987123 15342 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:10.987294 15342 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:10.995759 15342 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:10.995863 15342 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:10.995920 15342 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:10.995968 15342 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:11.007167 15342 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:11.008848 15342 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:11.010317 15342 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:11.018918 15347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:11.019192 15350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:11.018973 15348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:11.020814 15342 server_base.cc:1061] running on GCE node
I20260806 07:53:11.021677 15342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:11.023094 15342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:11.024339 15342 hybrid_clock.cc:648] HybridClock initialized: now 1786002791024299 us; error 51 us; skew 500 ppm
I20260806 07:53:11.024699 15342 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:11 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002791, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:11.028342 15342 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:53:11.030200 15342 webserver.cc:533] Webserver started at http://127.20.202.253:33653/ using document root <none> and password file <none>
I20260806 07:53:11.031090 15342 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:11.031245 15342 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:11.038082 15342 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:53:11.041044 15357 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:11.042590 15342 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:11.042778 15342 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "d633d87bb50447589745984b4100056c"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:53:11.043537 15342 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:11.070226 15342 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:11.074573 15342 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:11.074872 15342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:11.097045 15342 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:39499
I20260806 07:53:11.097050 15408 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:39499 every 8 connection(s)
I20260806 07:53:11.098848 15342 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:53:11.106331 15409 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:11.108342 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15342
I20260806 07:53:11.108547 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:53:11.110129 15409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Bootstrap starting.
I20260806 07:53:11.130726 15409 log.cc:826] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:11.142851 15409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:11.143482 15409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: Bootstrap complete.
I20260806 07:53:11.151472 15409 raft_consensus.cc:359] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.152647 15409 raft_consensus.cc:740] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d633d87bb50447589745984b4100056c, State: Initialized, Role: FOLLOWER
I20260806 07:53:11.153661 15409 consensus_queue.cc:260] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.154798 15414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:11.155195 15414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:11.156368 15409 sys_catalog.cc:565] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:11.168447 15425 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:53:11.168632 15425 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: loading cluster ID for follower catalog manager: success
I20260806 07:53:11.173214 15425 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: acquiring CA information for follower catalog manager: success
I20260806 07:53:11.176553 15425 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:11.200855 15342 master_runner.cc:187] Error getting master registration for 127.20.202.252:38561: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38561: connect: Connection refused (error 111),
I20260806 07:53:11.206722 15342 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:11.246048 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38561
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:11.617028 15432 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:11.617403 15432 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:11.617558 15432 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:11.626235 15432 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:11.626379 15432 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:11.626510 15432 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:11.626593 15432 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:11.637671 15432 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:11.639377 15432 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:11.640794 15432 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:11.649425 15437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:11.649469 15438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:11.650468 15440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:11.650970 15432 server_base.cc:1061] running on GCE node
I20260806 07:53:11.651804 15432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:11.653350 15432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:11.654632 15432 hybrid_clock.cc:648] HybridClock initialized: now 1786002791654550 us; error 73 us; skew 500 ppm
I20260806 07:53:11.655066 15432 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:11 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002791, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:11.658813 15432 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:53:11.660647 15432 webserver.cc:533] Webserver started at http://127.20.202.252:34793/ using document root <none> and password file <none>
I20260806 07:53:11.661433 15432 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:11.661597 15432 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:11.667400 15432 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:53:11.670387 15447 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:11.671834 15432 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:11.672006 15432 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b"
format_stamp: "Formatted at 2026-08-06 07:52:52 on dist-test-slave-g350"
I20260806 07:53:11.672742 15432 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:11.687832 15432 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:11.691870 15432 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:11.692157 15432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:11.713872 15432 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:38561
I20260806 07:53:11.713852 15498 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:38561 every 8 connection(s)
I20260806 07:53:11.715636 15432 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:53:11.718802 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15432
I20260806 07:53:11.719084 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:53:11.725517 15499 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:11.732537 15499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Bootstrap starting.
Aug 06 07:53:11 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:53:11.742679 15505 raft_consensus.cc:493] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:53:11.743064 15505 raft_consensus.cc:515] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.745760 15505 leader_election.cc:290] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d633d87bb50447589745984b4100056c (127.20.202.253:39499), 64e7ce51c5b2478288bc91b7fd7f2d1b (127.20.202.252:38561)
Aug 06 07:53:11 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002790, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:11.763424 15499 log.cc:826] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Log is configured to *not* fsync() on all Append() calls
Aug 06 07:53:11 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:11.770097 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82c11602542a4f7c97ecf42cb4a97678" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d633d87bb50447589745984b4100056c" is_pre_election: true
I20260806 07:53:11.770753 15384 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82c11602542a4f7c97ecf42cb4a97678 in term 1.
I20260806 07:53:11.771662 15272 leader_election.cc:304] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82c11602542a4f7c97ecf42cb4a97678, d633d87bb50447589745984b4100056c; no voters:
I20260806 07:53:11.772105 15505 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:53:11.772253 15505 raft_consensus.cc:493] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:53:11.772360 15505 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:53:11.777460 15505 raft_consensus.cc:515] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.778353 15505 leader_election.cc:290] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [CANDIDATE]: Term 2 election: Requested vote from peers d633d87bb50447589745984b4100056c (127.20.202.253:39499), 64e7ce51c5b2478288bc91b7fd7f2d1b (127.20.202.252:38561)
I20260806 07:53:11.779111 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82c11602542a4f7c97ecf42cb4a97678" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d633d87bb50447589745984b4100056c"
I20260806 07:53:11.779309 15384 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:53:11.782225 15499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:11.782919 15499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Bootstrap complete.
I20260806 07:53:11.784721 15384 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82c11602542a4f7c97ecf42cb4a97678 in term 2.
I20260806 07:53:11.785251 15272 leader_election.cc:304] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82c11602542a4f7c97ecf42cb4a97678, d633d87bb50447589745984b4100056c; no voters:
I20260806 07:53:11.785614 15505 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 FOLLOWER]: Leader election won for term 2
Aug 06 07:53:11 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:11.786247 15505 raft_consensus.cc:697] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 LEADER]: Becoming Leader. State: Replica: 82c11602542a4f7c97ecf42cb4a97678, State: Running, Role: LEADER
I20260806 07:53:11.786777 15505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.790856 15511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c11602542a4f7c97ecf42cb4a97678. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:11.791172 15511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:11.792734 15499 raft_consensus.cc:359] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.793720 15499 raft_consensus.cc:740] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e7ce51c5b2478288bc91b7fd7f2d1b, State: Initialized, Role: FOLLOWER
I20260806 07:53:11.794420 15499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:11.795831 15514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:11.796471 15514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:11.796985 15499 sys_catalog.cc:565] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:11.797035 15473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82c11602542a4f7c97ecf42cb4a97678" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b"
I20260806 07:53:11.797525 15473 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:53:11.798002 15513 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:53:11.801052 15513 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:53:11.802140 15513 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:53:11.802222 15513 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:53:11.803274 15474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82c11602542a4f7c97ecf42cb4a97678" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" is_pre_election: true
I20260806 07:53:11.803723 15513 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:53:11.803987 15473 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82c11602542a4f7c97ecf42cb4a97678 in term 2.
I20260806 07:53:11.804903 15513 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Loaded TSK: 0
I20260806 07:53:11.805902 15513 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:53:11.820842 15525 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:53:11.821064 15525 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: loading cluster ID for follower catalog manager: success
I20260806 07:53:11.825196 15525 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: acquiring CA information for follower catalog manager: success
I20260806 07:53:11.827986 15525 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:11.947773 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:12.274366 15384 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 2 FOLLOWER]: Refusing update from remote peer 82c11602542a4f7c97ecf42cb4a97678: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:53:12.275406 15511 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:53:12.279713 15533 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c11602542a4f7c97ecf42cb4a97678. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:12.279989 15533 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:12.281011 15505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:12.281257 15505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:12.282027 15473 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Refusing update from remote peer 82c11602542a4f7c97ecf42cb4a97678: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:53:12.282285 15533 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:12.282513 15533 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:12.282742 15511 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:53:12.297122 15539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:12.297648 15539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:12.298039 15514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c11602542a4f7c97ecf42cb4a97678. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:12.298332 15514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:53:12.310350 15532 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:12.310714 15532 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:12.310868 15532 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:12.319571 15532 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:12.319808 15532 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:12.330780 15532 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:12.332505 15532 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:12.333961 15532 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:12.344329 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:12.344632 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:12.344890 15545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:12.346302 15532 server_base.cc:1061] running on GCE node
I20260806 07:53:12.346824 15532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:12.347844 15532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:12.349088 15532 hybrid_clock.cc:648] HybridClock initialized: now 1786002792349024 us; error 49 us; skew 500 ppm
I20260806 07:53:12.349414 15532 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:12 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:12.353030 15532 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:12.354730 15532 webserver.cc:533] Webserver started at http://127.20.202.193:46823/ using document root <none> and password file <none>
I20260806 07:53:12.355558 15532 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:12.355717 15532 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:12.361550 15532 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:53:12.364326 15554 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:12.365911 15532 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:12.366117 15532 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:53:12.366888 15532 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:12.394090 15532 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:12.397948 15532 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:12.398308 15532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:12.399322 15532 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:12.400940 15532 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:12.401062 15532 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:12.401198 15532 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:12.401286 15532 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:12.438267 15532 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42077
I20260806 07:53:12.438359 15666 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42077 every 8 connection(s)
I20260806 07:53:12.439801 15532 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:53:12.449491 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15532
I20260806 07:53:12.449726 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:53:12 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:53:12 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:53:12 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:53:12.469091 15668 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:53:12.469259 15669 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:53:12.469564 15668 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:12.469563 15669 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:12.470737 15668 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:53:12.470968 15669 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:53:12.471189 15667 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:53:12.471315 15667 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:12.471835 15667 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:53:12.473533 15285 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:42077)
I20260806 07:53:12.473645 15464 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:42077)
I20260806 07:53:12.473722 15374 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:42077)
I20260806 07:53:12.475996 15285 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:47901
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:12.578579 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:12.731592 15511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "82c11602542a4f7c97ecf42cb4a97678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:12.731850 15511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: LEADER
W20260806 07:53:12.929466 15681 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:12.929899 15681 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:12.930061 15681 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:12.938522 15681 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:12.938762 15681 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:53:12.950148 15681 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:12.951792 15681 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:12.953227 15681 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:12.963333 15687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:12.963343 15688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:12.963492 15690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:12.964143 15681 server_base.cc:1061] running on GCE node
I20260806 07:53:12.964706 15681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:12.965695 15681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:12.966931 15681 hybrid_clock.cc:648] HybridClock initialized: now 1786002792966885 us; error 54 us; skew 500 ppm
I20260806 07:53:12.967253 15681 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:12 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:12.971027 15681 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:53:12.972661 15681 webserver.cc:533] Webserver started at http://127.20.202.194:42511/ using document root <none> and password file <none>
I20260806 07:53:12.973416 15681 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:12.973570 15681 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:12.979223 15681 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:53:12.982065 15697 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:12.983371 15681 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:53:12.983557 15681 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "5b98d06cc9a0440daf3add1863ecb4d5"
format_stamp: "Formatted at 2026-08-06 07:52:54 on dist-test-slave-g350"
I20260806 07:53:12.984275 15681 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:12.997397 15681 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:13.001179 15681 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:13.001470 15681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:13.002449 15681 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:13.004056 15681 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:13.004187 15681 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:13.004320 15681 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:13.004410 15681 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:13.039916 15681 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:35107
I20260806 07:53:13.039981 15810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:35107 every 8 connection(s)
I20260806 07:53:13.041909 15681 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:53:13.047688 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15681
I20260806 07:53:13.047907 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:13.074815 15811 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:53:13.075286 15811 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:13.076399 15811 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:53:13.078656 15464 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35107)
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002792, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:13.091245 15812 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:53:13.091436 15812 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:13.092021 15812 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:53:13.093140 15285 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35107)
I20260806 07:53:13.094152 15285 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:43757
I20260806 07:53:13.094527 15813 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:53:13.094682 15813 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:13.095168 15813 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:53:13.096418 15374 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35107)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:13.177273 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:13.478971 15668 heartbeater.cc:499] Master 127.20.202.254:43837 was elected leader, sending a full tablet report...
W20260806 07:53:13.527359 15825 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:13.527671 15825 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:13.527833 15825 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:13.536243 15825 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:13.536484 15825 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:53:13.547202 15825 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:13.548962 15825 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:13.550364 15825 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:13.560456 15831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:13.560484 15830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:13.561275 15833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:13.561371 15825 server_base.cc:1061] running on GCE node
I20260806 07:53:13.562004 15825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:13.562944 15825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:13.564201 15825 hybrid_clock.cc:648] HybridClock initialized: now 1786002793564159 us; error 58 us; skew 500 ppm
I20260806 07:53:13.564543 15825 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:13.568280 15825 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:53:13.570006 15825 webserver.cc:533] Webserver started at http://127.20.202.195:43285/ using document root <none> and password file <none>
I20260806 07:53:13.570756 15825 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:13.570909 15825 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:13.576689 15825 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 07:53:13.579483 15840 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:13.580909 15825 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:53:13.581089 15825 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "dadf93bef9884c7d98465809491d2626"
format_stamp: "Formatted at 2026-08-06 07:52:55 on dist-test-slave-g350"
I20260806 07:53:13.581874 15825 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:13.609977 15825 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:13.614614 15825 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:13.614917 15825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:13.615934 15825 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:13.617524 15825 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:13.617676 15825 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:53:13.617802 15825 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:13.617890 15825 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:13.652583 15825 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:39785
I20260806 07:53:13.652647 15952 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:39785 every 8 connection(s)
I20260806 07:53:13.654430 15825 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:53:13.655659 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15825
I20260806 07:53:13.655864 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:53:13 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002793, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:13.685859 15954 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:53:13.686281 15954 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:13.687355 15954 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:53:13.688781 15953 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:53:13.688922 15953 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:13.689244 15285 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:39785)
I20260806 07:53:13.689548 15953 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
I20260806 07:53:13.690416 15285 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:42325
I20260806 07:53:13.690616 15464 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:39785)
I20260806 07:53:13.692248 15955 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:53:13.692397 15955 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:13.692693 15955 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:53:13.693699 15374 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:39785)
I20260806 07:53:13.700047 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:53:13.707798 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:53:13.708029 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15253
I20260806 07:53:13.709100 15315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:13.828522 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15253
W20260806 07:53:13.851429 15360 connection.cc:573] server connection from 127.0.0.1:50766 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:53:13.852342 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--webserver_interface=127.20.202.254
--webserver_port=33689
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:14.098265 15812 heartbeater.cc:499] Master 127.20.202.254:43837 was elected leader, sending a full tablet report...
W20260806 07:53:14.102802 15812 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:43837 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:43837: connect: Connection refused (error 111)
W20260806 07:53:14.209179 15964 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:14.209532 15964 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:14.209724 15964 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:14.218310 15964 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:14.218410 15964 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:14.218519 15964 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:14.218604 15964 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:14.230574 15964 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=33689
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:14.232333 15964 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:14.234217 15964 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:14.243629 15971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:14.243571 15970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:14.246477 15973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:14.247099 15964 server_base.cc:1061] running on GCE node
I20260806 07:53:14.247861 15964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:14.249881 15964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:14.251174 15964 hybrid_clock.cc:648] HybridClock initialized: now 1786002794251053 us; error 98 us; skew 500 ppm
I20260806 07:53:14.251657 15964 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:14.257098 15964 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:53:14.259305 15964 webserver.cc:533] Webserver started at http://127.20.202.254:33689/ using document root <none> and password file <none>
I20260806 07:53:14.260125 15964 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:14.260313 15964 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:14.268306 15964 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:53:14.272606 15980 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:14.274534 15964 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:53:14.274711 15964 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "82c11602542a4f7c97ecf42cb4a97678"
format_stamp: "Formatted at 2026-08-06 07:52:51 on dist-test-slave-g350"
I20260806 07:53:14.275877 15964 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:14.298324 15964 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:14.302901 15964 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:14.303187 15964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:14.324615 15964 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43837
I20260806 07:53:14.324635 16031 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43837 every 8 connection(s)
I20260806 07:53:14.326606 15964 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:53:14.332064 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 15964
I20260806 07:53:14.332470 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15532
I20260806 07:53:14.335371 16032 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:14.339129 16032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap starting.
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:53:14.352953 15662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:14.359931 16032 log.cc:826] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:14.369539 16032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:14.370141 16032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: Bootstrap complete.
I20260806 07:53:14.377908 16032 raft_consensus.cc:359] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:14.378685 16032 raft_consensus.cc:740] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c11602542a4f7c97ecf42cb4a97678, State: Initialized, Role: FOLLOWER
I20260806 07:53:14.379442 16032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:14.380487 16038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:14.380791 16038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:14.381836 16032 sys_catalog.cc:565] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:14.393822 16049 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:53:14.393959 16049 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: loading cluster ID for follower catalog manager: success
I20260806 07:53:14.401806 16049 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: acquiring CA information for follower catalog manager: success
I20260806 07:53:14.405323 16049 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:53:14.447391 15964 master_runner.cc:264] No leader master found from master 82c11602542a4f7c97ecf42cb4a97678
I20260806 07:53:14.447541 15964 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:53:14.465126 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15532
I20260806 07:53:14.484629 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42077
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--webserver_port=46823
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:14.693296 15954 heartbeater.cc:499] Master 127.20.202.254:43837 was elected leader, sending a full tablet report...
I20260806 07:53:14.705273 15997 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dadf93bef9884c7d98465809491d2626" instance_seqno: 1786002793644056) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:33731; Asking this server to re-register.
I20260806 07:53:14.706283 15954 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:14.708083 15997 ts_manager.cc:194] Registered new tserver with Master: dadf93bef9884c7d98465809491d2626 (127.20.202.195:39785)
W20260806 07:53:14.852995 16055 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:14.853338 16055 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:14.853488 16055 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:14.862208 16055 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:14.862461 16055 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:14.873011 16055 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46823
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:14.874593 16055 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:14.875926 16055 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:14.886405 16061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:14.886376 16062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:14.887140 16064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:14.887993 16055 server_base.cc:1061] running on GCE node
I20260806 07:53:14.888590 16055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:14.889642 16055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:14.890969 16055 hybrid_clock.cc:648] HybridClock initialized: now 1786002794890874 us; error 56 us; skew 500 ppm
I20260806 07:53:14.891306 16055 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:14.895242 16055 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:14.896968 16055 webserver.cc:533] Webserver started at http://127.20.202.193:46823/ using document root <none> and password file <none>
I20260806 07:53:14.897819 16055 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:14.897964 16055 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:14.903542 16055 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:53:14.906394 16071 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:14.907760 16055 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:14.907939 16055 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:53:14.908680 16055 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:14.924515 16055 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:14.928639 16055 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:14.928938 16055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:14.929987 16055 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:14.931587 16055 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:14.931711 16055 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:14.931841 16055 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:14.931928 16055 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:14.968626 16055 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42077
I20260806 07:53:14.968725 16183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42077 every 8 connection(s)
I20260806 07:53:14.970279 16055 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:53:14.974756 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 16055
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:53:15.001701 16184 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38561
I20260806 07:53:15.002200 16184 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:15.003643 16184 heartbeater.cc:507] Master 127.20.202.252:38561 requested a full tablet report, sending...
Aug 06 07:53:14 dist-test-slave-g350 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002794, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:15.007290 15464 ts_manager.cc:194] Re-registered known tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:42077)
I20260806 07:53:15.009577 16185 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:53:15.009786 16185 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:15.010430 16185 heartbeater.cc:507] Master 127.20.202.254:43837 requested a full tablet report, sending...
I20260806 07:53:15.011739 15997 ts_manager.cc:194] Registered new tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:42077)
I20260806 07:53:15.015187 16186 heartbeater.cc:344] Connected to a master server at 127.20.202.253:39499
I20260806 07:53:15.015339 16186 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:15.015811 16186 heartbeater.cc:507] Master 127.20.202.253:39499 requested a full tablet report, sending...
I20260806 07:53:15.017127 15374 ts_manager.cc:194] Re-registered known tserver with Master: 67d01d5980424409ba090ddceddfe779 (127.20.202.193:42077)
I20260806 07:53:15.112932 15812 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43837
I20260806 07:53:15.113267 15812 heartbeater.cc:499] Master 127.20.202.254:43837 was elected leader, sending a full tablet report...
I20260806 07:53:15.114383 15997 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5b98d06cc9a0440daf3add1863ecb4d5" instance_seqno: 1786002793031041) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:41789; Asking this server to re-register.
I20260806 07:53:15.115134 15812 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:15.115343 16196 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 82c11602542a4f7c97ecf42cb4a97678)
I20260806 07:53:15.115581 16196 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:15.116506 15997 ts_manager.cc:194] Registered new tserver with Master: 5b98d06cc9a0440daf3add1863ecb4d5 (127.20.202.194:35107)
I20260806 07:53:15.118137 16196 leader_election.cc:290] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837), d633d87bb50447589745984b4100056c (127.20.202.253:39499)
I20260806 07:53:15.119422 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d633d87bb50447589745984b4100056c" is_pre_election: true
I20260806 07:53:15.124898 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "82c11602542a4f7c97ecf42cb4a97678" is_pre_election: true
I20260806 07:53:15.125579 16007 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64e7ce51c5b2478288bc91b7fd7f2d1b in term 2.
I20260806 07:53:15.126274 15451 leader_election.cc:304] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64e7ce51c5b2478288bc91b7fd7f2d1b, 82c11602542a4f7c97ecf42cb4a97678; no voters: d633d87bb50447589745984b4100056c
I20260806 07:53:15.126729 16196 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:53:15.126875 16196 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 82c11602542a4f7c97ecf42cb4a97678)
I20260806 07:53:15.127004 16196 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:53:15.129195 16196 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:15.129853 16196 leader_election.cc:290] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 3 election: Requested vote from peers 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837), d633d87bb50447589745984b4100056c (127.20.202.253:39499)
I20260806 07:53:15.130494 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d633d87bb50447589745984b4100056c"
I20260806 07:53:15.130456 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "82c11602542a4f7c97ecf42cb4a97678"
I20260806 07:53:15.130714 16007 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:53:15.136866 16007 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64e7ce51c5b2478288bc91b7fd7f2d1b in term 3.
I20260806 07:53:15.137355 15451 leader_election.cc:304] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64e7ce51c5b2478288bc91b7fd7f2d1b, 82c11602542a4f7c97ecf42cb4a97678; no voters: d633d87bb50447589745984b4100056c
I20260806 07:53:15.137838 16196 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:53:15.138372 16196 raft_consensus.cc:697] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [term 3 LEADER]: Becoming Leader. State: Replica: 64e7ce51c5b2478288bc91b7fd7f2d1b, State: Running, Role: LEADER
I20260806 07:53:15.138911 16196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
W20260806 07:53:15.140188 16200 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:15.136278 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.20.202.193:42077)
0806 07:53:15.137051 (+ 773us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:15.137060 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:15.137103 (+ 43us) server_negotiation.cc:408] Connection header received
0806 07:53:15.137180 (+ 77us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:15.137191 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:15.137455 (+ 264us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:15.137712 (+ 257us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:15.138740 (+ 1028us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48732: BlockingRecv error: recv got EOF from 127.0.0.1:48732 (error 108)
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":401,"threads_started":1}
I20260806 07:53:15.140733 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16055
I20260806 07:53:15.141366 16201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:15.141701 16201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:15.142549 16203 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
W20260806 07:53:15.143299 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16055: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:42077: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:53:15.143437 16200 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:15.141961 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.20.202.193:42077)
0806 07:53:15.142111 (+ 150us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:15.142115 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:15.142129 (+ 14us) server_negotiation.cc:408] Connection header received
0806 07:53:15.142169 (+ 40us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:15.142174 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:15.142225 (+ 51us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:15.142298 (+ 73us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:15.143194 (+ 896us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48734: BlockingRecv error: recv got EOF from 127.0.0.1:48734 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 07:53:15.143473 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 16055
I20260806 07:53:15.146519 16203 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:53:15.147384 16203 catalog_manager.cc:1295] Loaded cluster ID: 5ee07f79ee6046b2a5afedceb1bde610
I20260806 07:53:15.147503 16203 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:53:15.148722 16203 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:53:15.149786 16203 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b: Loaded TSK: 0
I20260806 07:53:15.150714 16203 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:53:15.162999 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42077
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--webserver_port=46823
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:15.330170 16205 raft_consensus.cc:493] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 82c11602542a4f7c97ecf42cb4a97678)
I20260806 07:53:15.330425 16205 raft_consensus.cc:515] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } }
I20260806 07:53:15.332546 16205 leader_election.cc:290] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 82c11602542a4f7c97ecf42cb4a97678 (127.20.202.254:43837), 64e7ce51c5b2478288bc91b7fd7f2d1b (127.20.202.252:38561)
I20260806 07:53:15.341890 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d633d87bb50447589745984b4100056c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "82c11602542a4f7c97ecf42cb4a97678" is_pre_election: true
I20260806 07:53:15.342367 16007 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d633d87bb50447589745984b4100056c in current term 3: Already voted for candidate 64e7ce51c5b2478288bc91b7fd7f2d1b in this term.
I20260806 07:53:15.346839 15473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d633d87bb50447589745984b4100056c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" is_pre_election: true
I20260806 07:53:15.347496 15361 leader_election.cc:304] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d633d87bb50447589745984b4100056c; no voters: 64e7ce51c5b2478288bc91b7fd7f2d1b, 82c11602542a4f7c97ecf42cb4a97678
I20260806 07:53:15.347955 16205 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:53:15.350302 16205 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260806 07:53:15.519629 16204 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:15.519989 16204 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:15.520152 16204 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:53:15.520260 16204 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:15.528818 16204 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:15.529093 16204 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:15.540174 16204 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46823
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:15.542013 16204 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:15.543459 16204 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:15.554145 16212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:15.554373 16213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:15.554580 16215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:15.555225 16204 server_base.cc:1061] running on GCE node
I20260806 07:53:15.555825 16204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:15.556833 16204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:15.558156 16204 hybrid_clock.cc:648] HybridClock initialized: now 1786002795558076 us; error 46 us; skew 500 ppm
I20260806 07:53:15.558519 16204 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:53:15.558964 15384 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [term 3 FOLLOWER]: Refusing update from remote peer 64e7ce51c5b2478288bc91b7fd7f2d1b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:53:15.559873 16201 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:53:15.562011 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:15.562281 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 07:53:15 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:15.563612 16204 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:15.564549 16196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:15.564890 16196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:15.565816 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:15.566048 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633d87bb50447589745984b4100056c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:15.566546 16007 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [term 3 FOLLOWER]: Refusing update from remote peer 64e7ce51c5b2478288bc91b7fd7f2d1b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:53:15.566838 16204 webserver.cc:533] Webserver started at http://127.20.202.193:46823/ using document root <none> and password file <none>
I20260806 07:53:15.567356 16196 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:53:15.567833 16204 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:15.567975 16204 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:15.573700 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:15.574167 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:15.574539 16224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e7ce51c5b2478288bc91b7fd7f2d1b. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:15.574785 16224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c11602542a4f7c97ecf42cb4a97678 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:15.576097 16204 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:53:15.579054 16229 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:15.580543 16204 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:15.580736 16204 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:53:15.581729 16204 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:15.614420 16204 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:15.621340 16204 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:15.621723 16204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:15.622812 16204 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:15.624609 16204 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:15.624686 16204 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:15.624853 16204 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:15.624923 16204 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:53:15.625322 16204 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:53:15.625415 16204 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:53:15.625519 16204 fault_injection.cc:59] injecting fault for kudu (pid 16204): FLAGS_fault_before_start (process will exit)
I20260806 07:53:15.644878 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42077
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--webserver_port=46823
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:15.999648 16237 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:15.999956 16237 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:16.000106 16237 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:16.008371 16237 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:16.008563 16237 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:16.019445 16237 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46823
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43837,127.20.202.253:39499,127.20.202.252:38561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:16.021200 16237 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:16.022535 16237 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:16.032485 16242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:16.032518 16243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:16.033705 16245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:16.034027 16237 server_base.cc:1061] running on GCE node
I20260806 07:53:16.034757 16237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:16.035758 16237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:16.036994 16237 hybrid_clock.cc:648] HybridClock initialized: now 1786002796036942 us; error 62 us; skew 500 ppm
I20260806 07:53:16.037325 16237 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:16 dist-test-slave-g350 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002796, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:16.041342 16237 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:16.043000 16196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c11602542a4f7c97ecf42cb4a97678" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43837 } } peers { permanent_uuid: "d633d87bb50447589745984b4100056c" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 39499 } } peers { permanent_uuid: "64e7ce51c5b2478288bc91b7fd7f2d1b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38561 } } }
I20260806 07:53:16.043241 16196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e7ce51c5b2478288bc91b7fd7f2d1b [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:16.044425 16237 webserver.cc:533] Webserver started at http://127.20.202.193:46823/ using document root <none> and password file <none>
I20260806 07:53:16.045254 16237 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:16.045356 16237 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:16.051524 16237 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:53:16.055060 16253 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:16.056552 16237 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:53:16.056737 16237 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "67d01d5980424409ba090ddceddfe779"
format_stamp: "Formatted at 2026-08-06 07:52:53 on dist-test-slave-g350"
I20260806 07:53:16.057551 16237 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:16.095561 16237 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:16.100306 16237 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:16.100603 16237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:16.101862 16237 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:16.103917 16237 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:16.104027 16237 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:16.104123 16237 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:16.104194 16237 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:16.139823 16237 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42077
I20260806 07:53:16.139904 16365 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42077 every 8 connection(s)
I20260806 07:53:16.141384 16237 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:53:16.145174 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 16237
I20260806 07:53:16.145533 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16237
W20260806 07:53:16.164273 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16237: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:42077: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:16.164494 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 16237
I20260806 07:53:16.179692 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15681
W20260806 07:53:16.180357 16375 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:16.147367 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:42709 (local address 127.20.202.252:38561)
0806 07:53:16.163235 (+ 15868us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:16.163241 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:16.179376 (+ 16135us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:42709: BlockingRecv error: recv got EOF from 127.20.202.193:42709 (error 108)
Metrics: {"server-negotiator.queue_time_us":15750,"thread_start_us":15640,"threads_started":1}
W20260806 07:53:16.180361 16370 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:16.144733 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:51247 (local address 127.20.202.253:39499)
0806 07:53:16.145357 (+ 624us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:16.145366 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:16.152042 (+ 6676us) server_negotiation.cc:408] Connection header received
0806 07:53:16.156181 (+ 4139us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:16.156190 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:16.156318 (+ 128us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:16.156457 (+ 139us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:16.163359 (+ 6902us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:53:16.165482 (+ 2123us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:53:16.179477 (+ 13995us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:51247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":347,"threads_started":1}
W20260806 07:53:16.183214 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15681: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:35107: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:16.183354 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15681
I20260806 07:53:16.202817 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15825
W20260806 07:53:16.205531 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15825: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:39785: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:16.205760 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15825
I20260806 07:53:16.224293 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15964
W20260806 07:53:16.226768 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15964: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:43837: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:16.226928 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15964
I20260806 07:53:16.249678 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15342
I20260806 07:53:16.251015 15404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:16.362203 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15342
I20260806 07:53:16.384670 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 15432
I20260806 07:53:16.386057 15494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:16.495797 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 15432
Shutting down hive metastore.
2026-08-06T07:53:16Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/10 (44952 ms)
[ RUN ] ExternalMiniClusterTest.TestBasicOperation/11
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:53:17 dist-test-slave-g350 krb5kdc[16387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:53:17 dist-test-slave-g350 krb5kdc[16387](info): set up 2 sockets
Aug 06 07:53:17 dist-test-slave-g350 krb5kdc[16387](info): commencing operation
krb5kdc: starting...
W20260806 07:53:19.085495 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.159s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:53:19 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:53:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:19Z Disabled control of system clock
2026-08-06T07:53:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:19Z Disabled control of system clock
2026-08-06T07:53:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:19Z Disabled control of system clock
2026-08-06T07:53:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:19Z Disabled control of system clock
2026-08-06T07:53:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:19Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:53:26: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:53:29 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002809, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:53:35 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002815, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:53:35 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002815, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:37.542938 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:37.893852 16690 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:37.894160 16690 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:37.894243 16690 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:37.902535 16690 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:37.902648 16690 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:37.902707 16690 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:37.902756 16690 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:37.913792 16690 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:37.915504 16690 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:37.917065 16690 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:37.926234 16695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:37.926265 16696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:37.927045 16698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:37.927774 16690 server_base.cc:1061] running on GCE node
I20260806 07:53:37.928536 16690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:37.930322 16690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:37.931596 16690 hybrid_clock.cc:648] HybridClock initialized: now 1786002817931437 us; error 124 us; skew 500 ppm
I20260806 07:53:37.931967 16690 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:37 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:37.935109 16690 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:53:37.936627 16690 webserver.cc:533] Webserver started at http://127.20.202.254:46651/ using document root <none> and password file <none>
I20260806 07:53:37.937314 16690 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:37.937407 16690 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:37.937772 16690 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:53:37.940711 16690 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "db950c71d6d14609b7f545c7d85763d4"
format_stamp: "Formatted at 2026-08-06 07:53:37 on dist-test-slave-g350"
I20260806 07:53:37.941880 16690 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "db950c71d6d14609b7f545c7d85763d4"
format_stamp: "Formatted at 2026-08-06 07:53:37 on dist-test-slave-g350"
I20260806 07:53:37.948655 16690 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 07:53:37.952332 16705 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:37.953930 16690 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:53:37.954169 16690 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "db950c71d6d14609b7f545c7d85763d4"
format_stamp: "Formatted at 2026-08-06 07:53:37 on dist-test-slave-g350"
I20260806 07:53:37.954356 16690 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:37.971330 16690 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:37.975405 16690 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:37.975714 16690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:37.996619 16690 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34961
I20260806 07:53:37.996721 16756 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34961 every 8 connection(s)
I20260806 07:53:37.998307 16690 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:53:38.002610 16757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:53:38.003275 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 16690
I20260806 07:53:38.003491 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:53:38.012075 16757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:53:38 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
I20260806 07:53:38.038568 16757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:53:38.040020 16707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111)
W20260806 07:53:38.041728 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:41389: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:38.091117 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } attempt: 1
W20260806 07:53:38.093426 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:41389: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:38.141891 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:41389
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:38.161746 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } attempt: 2
W20260806 07:53:38.163810 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:41389: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111)
I20260806 07:53:38.255096 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } attempt: 3
W20260806 07:53:38.257341 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:41389: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111)
I20260806 07:53:38.400636 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } attempt: 4
W20260806 07:53:38.403093 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:41389: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111)
W20260806 07:53:38.501632 16766 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:38.502061 16766 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:38.502228 16766 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:38.510684 16766 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:38.510846 16766 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:38.510975 16766 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:38.511060 16766 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:38.522778 16766 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:38.524578 16766 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:38.526139 16766 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:38.535207 16772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:38.535295 16771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:38.536073 16774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:38.537185 16766 server_base.cc:1061] running on GCE node
I20260806 07:53:38.538272 16766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:38.540282 16766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:38.541652 16766 hybrid_clock.cc:648] HybridClock initialized: now 1786002818541442 us; error 115 us; skew 500 ppm
I20260806 07:53:38.542156 16766 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:38 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:38.546018 16766 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:53:38.548589 16766 webserver.cc:533] Webserver started at http://127.20.202.253:41185/ using document root <none> and password file <none>
I20260806 07:53:38.549773 16766 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:38.549953 16766 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:38.550393 16766 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:53:38.553445 16766 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "def6bb4233af46a287e75eed9c6b97cf"
format_stamp: "Formatted at 2026-08-06 07:53:38 on dist-test-slave-g350"
I20260806 07:53:38.554818 16766 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "def6bb4233af46a287e75eed9c6b97cf"
format_stamp: "Formatted at 2026-08-06 07:53:38 on dist-test-slave-g350"
I20260806 07:53:38.562120 16766 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 07:53:38.566165 16781 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:38.567763 16766 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:53:38.567981 16766 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "def6bb4233af46a287e75eed9c6b97cf"
format_stamp: "Formatted at 2026-08-06 07:53:38 on dist-test-slave-g350"
I20260806 07:53:38.568240 16766 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:38.604331 16766 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:38.608510 16766 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:38.608851 16766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:38.630744 16766 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:41389
I20260806 07:53:38.630785 16833 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:41389 every 8 connection(s)
I20260806 07:53:38.632475 16766 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:53:38.635396 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 16766
I20260806 07:53:38.635676 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:53:38.638108 16834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:53:38.647469 16834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:53:38 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
I20260806 07:53:38.671998 16834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:53:38 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:38.691954 16834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:53:38.693281 16783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
W20260806 07:53:38.695256 16834 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:38.702351 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Aug 06 07:53:38 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:38.717010 16757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:53:38.719138 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:53:38.744635 16834 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 1
W20260806 07:53:38.747324 16834 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:38.770394 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
W20260806 07:53:38.772913 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:38.774333 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41411
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:38.815598 16834 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 2
W20260806 07:53:38.817996 16834 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:38.841221 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 2
W20260806 07:53:38.843916 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:38.909356 16834 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 3
W20260806 07:53:38.911934 16834 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:38.950286 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 3
W20260806 07:53:38.952905 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:39.055286 16834 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 4
W20260806 07:53:39.058157 16834 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
I20260806 07:53:39.130177 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 4
W20260806 07:53:39.132470 16757 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111)
W20260806 07:53:39.148808 16844 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:39.149205 16844 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:39.149366 16844 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:39.158289 16844 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:39.158406 16844 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:39.158491 16844 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:39.158572 16844 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:39.171425 16844 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:39.173161 16844 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:39.174757 16844 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:39.183466 16850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:39.183918 16849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:39.185319 16852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:39.185567 16844 server_base.cc:1061] running on GCE node
I20260806 07:53:39.186451 16844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:39.188095 16844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:39.189478 16844 hybrid_clock.cc:648] HybridClock initialized: now 1786002819189297 us; error 153 us; skew 500 ppm
I20260806 07:53:39.189882 16844 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002819, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:39.193387 16844 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:53:39.195266 16844 webserver.cc:533] Webserver started at http://127.20.202.252:45171/ using document root <none> and password file <none>
I20260806 07:53:39.196146 16844 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:39.196257 16844 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:39.196648 16844 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:53:39.199443 16844 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "e122a8d66b7d4114adbc03c6be597e4f"
format_stamp: "Formatted at 2026-08-06 07:53:39 on dist-test-slave-g350"
I20260806 07:53:39.200335 16844 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "e122a8d66b7d4114adbc03c6be597e4f"
format_stamp: "Formatted at 2026-08-06 07:53:39 on dist-test-slave-g350"
I20260806 07:53:39.206377 16844 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 07:53:39.210662 16859 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:39.212334 16844 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:53:39.212566 16844 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "e122a8d66b7d4114adbc03c6be597e4f"
format_stamp: "Formatted at 2026-08-06 07:53:39 on dist-test-slave-g350"
I20260806 07:53:39.213016 16844 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:39.241679 16844 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:39.246346 16844 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:39.246690 16844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:39.268327 16844 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:41411
I20260806 07:53:39.268352 16911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:41411 every 8 connection(s)
I20260806 07:53:39.269975 16844 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:53:39.274622 16912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:53:39.275939 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 16844
I20260806 07:53:39.276161 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:53:39.285413 16912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002819, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:39.309278 16912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002819, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:39.325739 16912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002819, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:39.345837 16912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Bootstrap starting.
I20260806 07:53:39.349042 16912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Neither blocks nor log segments found. Creating new log.
I20260806 07:53:39.350550 16912 log.cc:826] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:39.353324 16912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: No bootstrap required, opened a new log
I20260806 07:53:39.357445 16834 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 5
I20260806 07:53:39.359957 16912 raft_consensus.cc:359] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.360435 16912 raft_consensus.cc:385] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:53:39.360605 16912 raft_consensus.cc:740] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e122a8d66b7d4114adbc03c6be597e4f, State: Initialized, Role: FOLLOWER
I20260806 07:53:39.361306 16912 consensus_queue.cc:260] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.362361 16923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.362653 16923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.363976 16912 sys_catalog.cc:565] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
W20260806 07:53:39.380587 16934 catalog_manager.cc:1594] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:53:39.380877 16934 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:53:39.372123 16844 master_runner.cc:187] Error getting master registration for 127.20.202.254:34961: OK, instance_id { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" instance_seqno: 1786002817990914 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:53:39.383460 16844 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:53:39.385124 16834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Bootstrap starting.
I20260806 07:53:39.389318 16834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Neither blocks nor log segments found. Creating new log.
I20260806 07:53:39.390465 16834 log.cc:826] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:39.393051 16834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: No bootstrap required, opened a new log
I20260806 07:53:39.399461 16834 raft_consensus.cc:359] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.399781 16834 raft_consensus.cc:385] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:53:39.399947 16834 raft_consensus.cc:740] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: def6bb4233af46a287e75eed9c6b97cf, State: Initialized, Role: FOLLOWER
I20260806 07:53:39.400609 16834 consensus_queue.cc:260] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.401695 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.402004 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.402910 16834 sys_catalog.cc:565] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:39.409730 16757 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } attempt: 5
I20260806 07:53:39.413015 16937 raft_consensus.cc:493] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:53:39.413278 16937 raft_consensus.cc:515] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.415217 16937 leader_election.cc:290] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db950c71d6d14609b7f545c7d85763d4 (127.20.202.254:34961), e122a8d66b7d4114adbc03c6be597e4f (127.20.202.252:41411)
Aug 06 07:53:39 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:39.418584 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e122a8d66b7d4114adbc03c6be597e4f" is_pre_election: true
I20260806 07:53:39.419322 16886 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate def6bb4233af46a287e75eed9c6b97cf in term 0.
I20260806 07:53:39.421532 16783 leader_election.cc:304] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: def6bb4233af46a287e75eed9c6b97cf, e122a8d66b7d4114adbc03c6be597e4f; no voters:
I20260806 07:53:39.422049 16937 raft_consensus.cc:2804] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:53:39.422202 16937 raft_consensus.cc:493] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:53:39.422304 16937 raft_consensus.cc:3060] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:53:39.426654 16937 raft_consensus.cc:515] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.413328 16766 master_runner.cc:187] Error getting master registration for 127.20.202.254:34961: OK, instance_id { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" instance_seqno: 1786002817990914 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:53:39.427820 16766 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:53:39.431775 16937 leader_election.cc:290] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 1 election: Requested vote from peers db950c71d6d14609b7f545c7d85763d4 (127.20.202.254:34961), e122a8d66b7d4114adbc03c6be597e4f (127.20.202.252:41411)
I20260806 07:53:39.431874 16886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e122a8d66b7d4114adbc03c6be597e4f"
I20260806 07:53:39.432101 16886 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:53:39.433156 16757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap starting.
I20260806 07:53:39.435923 16886 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate def6bb4233af46a287e75eed9c6b97cf in term 1.
I20260806 07:53:39.436534 16783 leader_election.cc:304] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: def6bb4233af46a287e75eed9c6b97cf, e122a8d66b7d4114adbc03c6be597e4f; no voters:
I20260806 07:53:39.437114 16937 raft_consensus.cc:2804] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:53:39.437656 16757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Neither blocks nor log segments found. Creating new log.
I20260806 07:53:39.439363 16757 log.cc:826] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:39.440420 16937 raft_consensus.cc:697] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 LEADER]: Becoming Leader. State: Replica: def6bb4233af46a287e75eed9c6b97cf, State: Running, Role: LEADER
I20260806 07:53:39.441867 16937 consensus_queue.cc:237] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.446683 16757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: No bootstrap required, opened a new log
W20260806 07:53:39.437116 16948 catalog_manager.cc:1594] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:53:39.447302 16948 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:53:39.457094 16757 raft_consensus.cc:359] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.458689 16950 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.459160 16950 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:39.460914 16953 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:53:39.457531 16757 raft_consensus.cc:385] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:53:39.462404 16757 raft_consensus.cc:740] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db950c71d6d14609b7f545c7d85763d4, State: Initialized, Role: FOLLOWER
I20260806 07:53:39.463366 16757 consensus_queue.cc:260] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:39.466677 16953 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:53:39.470616 16954 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.473243 16954 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.473161 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db950c71d6d14609b7f545c7d85763d4"
I20260806 07:53:39.473567 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db950c71d6d14609b7f545c7d85763d4" is_pre_election: true
I20260806 07:53:39.473707 16731 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:53:39.473156 16757 sys_catalog.cc:565] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:39.478864 16731 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate def6bb4233af46a287e75eed9c6b97cf in term 1.
W20260806 07:53:39.478883 16711 tablet.cc:2406] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:53:39.499164 16865 tablet.cc:2406] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:53:39.500768 16886 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 1 FOLLOWER]: Refusing update from remote peer def6bb4233af46a287e75eed9c6b97cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:53:39.500993 16731 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Refusing update from remote peer def6bb4233af46a287e75eed9c6b97cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:53:39.502250 16950 consensus_queue.cc:1048] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:53:39.502964 16937 consensus_queue.cc:1048] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:53:39.531350 16954 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.531710 16954 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.537783 16954 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.538038 16954 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.543176 16923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.543464 16923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.545712 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.545989 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:39.547890 16953 catalog_manager.cc:1383] Generated new cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:39.548004 16953 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:53:39.558046 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.558449 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:39.567907 16923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:39.568183 16923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:39.582649 16953 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:53:39.584486 16953 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:53:39.606683 16953 catalog_manager.cc:6070] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Generated new TSK 0
I20260806 07:53:39.607965 16953 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:53:39.614984 16974 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:39.615191 16974 catalog_manager.cc:1588] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: loading cluster ID for follower catalog manager: success
I20260806 07:53:39.619797 16974 catalog_manager.cc:1610] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: acquiring CA information for follower catalog manager: success
I20260806 07:53:39.622803 16974 catalog_manager.cc:1638] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:39.798398 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:40.255213 16981 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:40.255568 16981 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:40.255746 16981 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:40.264261 16981 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:40.264506 16981 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:40.280289 16981 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:40.282395 16981 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:40.284106 16981 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:40.294126 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:40.294193 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:40.295586 16989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:40.296008 16981 server_base.cc:1061] running on GCE node
I20260806 07:53:40.296540 16981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:40.297637 16981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:40.298888 16981 hybrid_clock.cc:648] HybridClock initialized: now 1786002820298722 us; error 130 us; skew 500 ppm
I20260806 07:53:40.299228 16981 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:40 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:40.302495 16981 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:40.304234 16981 webserver.cc:533] Webserver started at http://127.20.202.193:46633/ using document root <none> and password file <none>
I20260806 07:53:40.304966 16981 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:40.305112 16981 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:40.305438 16981 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:53:40.308015 16981 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:40.308807 16981 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:40.314280 16981 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 07:53:40.318276 16996 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:40.320221 16981 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:53:40.320423 16981 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:40.320660 16981 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:40.338896 16981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:40.343111 16981 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:40.343447 16981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:40.344524 16981 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:40.346236 16981 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:40.346315 16981 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:40.346459 16981 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:40.346547 16981 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:40.383356 16934 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:40.383507 16934 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: loading cluster ID for follower catalog manager: success
I20260806 07:53:40.385011 16981 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38061
I20260806 07:53:40.385059 17108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38061 every 8 connection(s)
I20260806 07:53:40.386512 16981 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:53:40.386834 16934 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: acquiring CA information for follower catalog manager: success
I20260806 07:53:40.389513 16934 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:53:40.391316 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 16981
I20260806 07:53:40.391588 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:53:40 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:53:40 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:40 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:40.443257 17110 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:53:40.443400 17111 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:53:40.443441 17109 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:53:40.443761 17109 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:40.443761 17111 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:40.443763 17110 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:40.445232 17109 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:53:40.445328 17110 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:53:40.445299 17111 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
I20260806 07:53:40.448226 16722 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:38061)
I20260806 07:53:40.448227 16798 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:38061)
I20260806 07:53:40.448531 16876 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:38061)
I20260806 07:53:40.450347 16798 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:53703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:40.548867 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:40.904876 17124 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:40.905256 17124 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:40.905411 17124 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:40.918497 17124 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:40.918789 17124 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:53:40.931708 17124 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:40.933488 17124 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:40.935099 17124 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:40.945731 17129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:40.946545 17130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:40.947110 17132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:40.947926 17124 server_base.cc:1061] running on GCE node
I20260806 07:53:40.948632 17124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:40.949980 17124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:40.951249 17124 hybrid_clock.cc:648] HybridClock initialized: now 1786002820951108 us; error 105 us; skew 500 ppm
I20260806 07:53:40.951635 17124 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:40 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:40.955358 17124 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:53:40.957182 17124 webserver.cc:533] Webserver started at http://127.20.202.194:33805/ using document root <none> and password file <none>
I20260806 07:53:40.958078 17124 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:40.958223 17124 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:40.958600 17124 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:53:40.961277 17124 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "dda75ebc05424ff48d8dc725bf7468f9"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:40.962177 17124 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "dda75ebc05424ff48d8dc725bf7468f9"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:40.968039 17124 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:53:40.971550 17139 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:40.973083 17124 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:53:40.973313 17124 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "dda75ebc05424ff48d8dc725bf7468f9"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:40.973587 17124 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:41.006594 17124 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:41.011080 17124 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:41.011332 17124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:41.012259 17124 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:41.013952 17124 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:41.014050 17124 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:41.014183 17124 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:41.014232 17124 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:41.050187 17124 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:40449
I20260806 07:53:41.050267 17251 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:40449 every 8 connection(s)
I20260806 07:53:41.051669 17124 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:53:41.060510 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 17124
I20260806 07:53:41.060760 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
I20260806 07:53:41.105491 17252 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:53:41.105443 17253 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:53:41.105937 17252 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:41.105943 17253 heartbeater.cc:461] Registering TS with master...
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002820, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.195@KRBTEST.COM" created.
I20260806 07:53:41.107173 17252 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:53:41.107204 17253 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:53:41.109283 16722 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:40449)
I20260806 07:53:41.110155 16876 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:40449)
I20260806 07:53:41.115530 17254 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:53:41.115722 17254 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:41.116284 17254 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:53:41.117398 16798 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:40449)
I20260806 07:53:41.118347 16798 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:50013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:41.207202 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:41.452850 17111 heartbeater.cc:499] Master 127.20.202.253:41389 was elected leader, sending a full tablet report...
W20260806 07:53:41.564944 17268 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:41.565315 17268 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:41.565490 17268 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:41.574254 17268 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:41.574483 17268 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:53:41.585960 17268 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:41.587689 17268 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:41.589025 17268 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:41.599331 17273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:41.599300 17274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:41.600153 17276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:41.600944 17268 server_base.cc:1061] running on GCE node
I20260806 07:53:41.601579 17268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:41.602818 17268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:41.604122 17268 hybrid_clock.cc:648] HybridClock initialized: now 1786002821603991 us; error 98 us; skew 500 ppm
I20260806 07:53:41.604614 17268 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:41.608228 17268 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:53:41.609977 17268 webserver.cc:533] Webserver started at http://127.20.202.195:33691/ using document root <none> and password file <none>
I20260806 07:53:41.610750 17268 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:41.610903 17268 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:41.611270 17268 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:53:41.613838 17268 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "3f3e95a8d2e44bab9356973be5491835"
format_stamp: "Formatted at 2026-08-06 07:53:41 on dist-test-slave-g350"
I20260806 07:53:41.614686 17268 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "3f3e95a8d2e44bab9356973be5491835"
format_stamp: "Formatted at 2026-08-06 07:53:41 on dist-test-slave-g350"
I20260806 07:53:41.620782 17268 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 07:53:41.624611 17283 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:41.626369 17268 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:53:41.626549 17268 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "3f3e95a8d2e44bab9356973be5491835"
format_stamp: "Formatted at 2026-08-06 07:53:41 on dist-test-slave-g350"
I20260806 07:53:41.626775 17268 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:41.648705 17268 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:41.653136 17268 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:41.653479 17268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:41.654592 17268 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:41.656459 17268 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:41.656584 17268 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:41.656749 17268 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:41.656828 17268 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:41.695128 17268 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:34625
I20260806 07:53:41.695180 17395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:34625 every 8 connection(s)
I20260806 07:53:41.696647 17268 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:53:41.697268 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 17268
I20260806 07:53:41.697557 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:41 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:41.741266 17396 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:53:41.741434 17398 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:53:41.741756 17398 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:41.741757 17396 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:41.742705 17397 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:53:41.742865 17397 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:41.743026 17396 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:53:41.743067 17398 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:53:41.743439 17397 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:53:41.745091 16876 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:34625)
I20260806 07:53:41.745290 16721 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:34625)
I20260806 07:53:41.745752 16798 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:34625)
I20260806 07:53:41.746592 16798 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:36479
I20260806 07:53:41.763245 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:53:41.771311 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:53:41.771561 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16690
I20260806 07:53:41.772485 16752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:41.883074 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 16690
I20260806 07:53:41.906935 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--webserver_interface=127.20.202.254
--webserver_port=46651
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:42.112782 17253 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:34961 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:34961: connect: Connection refused (error 111)
I20260806 07:53:42.121148 17254 heartbeater.cc:499] Master 127.20.202.253:41389 was elected leader, sending a full tablet report...
W20260806 07:53:42.162277 16784 consensus_peers.cc:597] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf -> Peer db950c71d6d14609b7f545c7d85763d4 (127.20.202.254:34961): Couldn't send request to peer db950c71d6d14609b7f545c7d85763d4. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:34961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:53:42.281693 17409 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:42.282064 17409 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:42.282164 17409 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:42.291548 17409 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:42.291675 17409 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:42.291736 17409 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:42.291810 17409 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:42.303443 17409 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=46651
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:42.305182 17409 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:42.306816 17409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:42.315604 17418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:42.315876 17420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:42.315632 17417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:42.317286 17409 server_base.cc:1061] running on GCE node
I20260806 07:53:42.318363 17409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:42.320482 17409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:42.321807 17409 hybrid_clock.cc:648] HybridClock initialized: now 1786002822321645 us; error 127 us; skew 500 ppm
I20260806 07:53:42.322218 17409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:42 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:42.326085 17409 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:53:42.327872 17409 webserver.cc:533] Webserver started at http://127.20.202.254:46651/ using document root <none> and password file <none>
I20260806 07:53:42.328671 17409 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:42.328827 17409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:42.335614 17409 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:53:42.338884 17427 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:42.340332 17409 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:53:42.340513 17409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "db950c71d6d14609b7f545c7d85763d4"
format_stamp: "Formatted at 2026-08-06 07:53:37 on dist-test-slave-g350"
I20260806 07:53:42.341253 17409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:42.356166 17409 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:42.361224 17409 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:42.361531 17409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:42.384465 17409 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34961
I20260806 07:53:42.384527 17478 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34961 every 8 connection(s)
I20260806 07:53:42.386436 17409 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:53:42.388034 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 17409
I20260806 07:53:42.388444 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16981
I20260806 07:53:42.395445 17479 sys_catalog.cc:263] Verifying existing consensus state
Aug 06 07:53:42 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:53:42.400138 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap starting.
I20260806 07:53:42.409183 17104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:42.425350 17479 log.cc:826] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:42.438084 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:42.438817 17479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap complete.
I20260806 07:53:42.446978 17479 raft_consensus.cc:359] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:42.447803 17479 raft_consensus.cc:740] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db950c71d6d14609b7f545c7d85763d4, State: Initialized, Role: FOLLOWER
I20260806 07:53:42.448697 17479 consensus_queue.cc:260] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:42.449782 17485 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:42.450112 17485 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:42.451057 17479 sys_catalog.cc:565] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:42.462765 17496 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:42.462929 17496 catalog_manager.cc:1588] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: loading cluster ID for follower catalog manager: success
I20260806 07:53:42.467687 17496 catalog_manager.cc:1610] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: acquiring CA information for follower catalog manager: success
I20260806 07:53:42.470613 17496 catalog_manager.cc:1638] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:53:42.524636 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 16981
W20260806 07:53:42.547286 16785 connection.cc:573] server connection from 127.20.202.193:53703 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:53:42.548197 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38061
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--webserver_port=46633
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:42.548393 17499 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:42.524272 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:34503 (local address 127.20.202.254:34961)
0806 07:53:42.524513 (+ 241us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:42.524520 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:42.547051 (+ 22531us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:34503: BlockingRecv error: recv got EOF from 127.20.202.193:34503 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 07:53:42.665407 17407 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:42.665760 17407 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:42.681192 17485 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:42.681543 17485 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:42.749239 17398 heartbeater.cc:499] Master 127.20.202.253:41389 was elected leader, sending a full tablet report...
I20260806 07:53:42.762809 17444 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f3e95a8d2e44bab9356973be5491835" instance_seqno: 1786002821686434) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:49833; Asking this server to re-register.
I20260806 07:53:42.763940 17397 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:42.766067 17444 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:34625)
W20260806 07:53:42.932185 17503 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:42.932555 17503 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:42.932706 17503 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:42.941385 17503 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:42.941699 17503 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:42.953016 17503 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46633
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:42.954855 17503 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:42.956341 17503 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:42.967160 17513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:42.967160 17514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:42.968017 17516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:42.968700 17503 server_base.cc:1061] running on GCE node
I20260806 07:53:42.969405 17503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:42.970685 17503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:42.971931 17503 hybrid_clock.cc:648] HybridClock initialized: now 1786002822971794 us; error 105 us; skew 500 ppm
I20260806 07:53:42.972337 17503 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:42 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:42.976284 17503 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:42.978099 17503 webserver.cc:533] Webserver started at http://127.20.202.193:46633/ using document root <none> and password file <none>
I20260806 07:53:42.979013 17503 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:42.979161 17503 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:42.984994 17503 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 07:53:42.988200 17523 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:42.989955 17503 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:53:42.990190 17503 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:42.990979 17503 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:43.007498 17503 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:43.011823 17503 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:43.012126 17503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:43.013126 17503 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:43.014875 17503 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:43.015017 17503 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:43.015146 17503 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:43.015234 17503 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:43.053797 17503 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38061
I20260806 07:53:43.053840 17636 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38061 every 8 connection(s)
I20260806 07:53:43.055465 17503 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:53:43.060637 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 17503
Aug 06 07:53:43 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:53:43 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:53:43 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:53:43 dist-test-slave-g350 krb5kdc[16387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002822, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:53:43.100873 17638 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:53:43.101358 17638 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:43.102568 17638 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:53:43.105046 17637 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:53:43.105195 17637 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:43.105289 17444 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:38061)
I20260806 07:53:43.105779 17637 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:53:43.107324 16876 ts_manager.cc:194] Re-registered known tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:38061)
I20260806 07:53:43.108911 17639 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:53:43.109124 17639 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:43.109709 17639 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:53:43.111245 16798 ts_manager.cc:194] Re-registered known tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:38061)
I20260806 07:53:43.112082 16798 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:51635
I20260806 07:53:43.122632 17253 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:53:43.124071 17444 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dda75ebc05424ff48d8dc725bf7468f9" instance_seqno: 1786002821041582) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:50541; Asking this server to re-register.
I20260806 07:53:43.124873 17253 heartbeater.cc:461] Registering TS with master...
I20260806 07:53:43.126165 17444 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:40449)
I20260806 07:53:43.238914 17504 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:43.239198 17504 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:43.240361 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17503
W20260806 07:53:43.241851 17653 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:43.237100 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.20.202.193:38061)
0806 07:53:43.237984 (+ 884us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:43.237994 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:43.238040 (+ 46us) server_negotiation.cc:408] Connection header received
0806 07:53:43.238121 (+ 81us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:43.238129 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:43.238467 (+ 338us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:43.238711 (+ 244us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:43.240209 (+ 1498us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45582: BlockingRecv error: recv got EOF from 127.0.0.1:45582 (error 108)
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":524,"threads_started":1}
W20260806 07:53:43.243115 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17503: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:38061: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:43.243268 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 17503
I20260806 07:53:43.262492 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38061
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--webserver_port=46633
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:43.624893 17658 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:43.625247 17658 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:43.625408 17658 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:53:43.625504 17658 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:43.634027 17658 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:43.634311 17658 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:43.645524 17658 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46633
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:43.647456 17658 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:43.648855 17658 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:43.659454 17664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:43.659623 17663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:43.659406 17666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:43.660516 17658 server_base.cc:1061] running on GCE node
I20260806 07:53:43.661187 17658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:43.662364 17658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:43.663661 17658 hybrid_clock.cc:648] HybridClock initialized: now 1786002823663472 us; error 140 us; skew 500 ppm
I20260806 07:53:43.663968 17658 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:43 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002823, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:43.667508 17658 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:43.669296 17658 webserver.cc:533] Webserver started at http://127.20.202.193:46633/ using document root <none> and password file <none>
I20260806 07:53:43.670440 17658 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:43.670557 17658 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:43.676254 17658 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:53:43.679479 17673 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:43.681104 17658 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:53:43.681320 17658 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:43.682206 17658 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:43.698477 17658 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:43.702342 17658 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:43.702632 17658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:43.703644 17658 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:43.705361 17658 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:43.705436 17658 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:43.705602 17658 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:43.705657 17658 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
W20260806 07:53:43.706105 17658 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:53:43.706193 17658 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:53:43.706278 17658 fault_injection.cc:59] injecting fault for kudu (pid 17658): FLAGS_fault_before_start (process will exit)
I20260806 07:53:43.728416 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:38061
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--webserver_port=46633
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:44.094803 17681 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:44.095196 17681 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:44.095360 17681 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:44.103953 17681 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:44.104213 17681 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:53:44.115343 17681 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35967,127.20.202.213:36337,127.20.202.214:35195,127.20.202.215:35107,127.20.202.216:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:38061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=46633
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:44.117381 17681 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:44.118999 17681 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:44.131219 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:44.131031 17686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:44.131296 17687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:44.132768 17681 server_base.cc:1061] running on GCE node
I20260806 07:53:44.133385 17681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:44.134743 17681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:44.135998 17681 hybrid_clock.cc:648] HybridClock initialized: now 1786002824135835 us; error 128 us; skew 500 ppm
I20260806 07:53:44.136386 17681 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:44 dist-test-slave-g350 krb5kdc[16387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002824, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:44.140038 17681 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:53:44.141906 17681 webserver.cc:533] Webserver started at http://127.20.202.193:46633/ using document root <none> and password file <none>
I20260806 07:53:44.142688 17681 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:44.142835 17681 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:44.148958 17681 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 07:53:44.152212 17696 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:44.153847 17681 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:44.154037 17681 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:53:44.154817 17681 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:44.199204 17681 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:44.203905 17681 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:44.204202 17681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:44.205341 17681 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:53:44.207311 17681 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:53:44.207445 17681 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:44.207573 17681 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:53:44.207664 17681 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:44.245476 17681 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38061
I20260806 07:53:44.245638 17810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38061 every 8 connection(s)
I20260806 07:53:44.247366 17681 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:53:44.251427 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 17681
I20260806 07:53:44.251811 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17681
W20260806 07:53:44.287645 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17681: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:38061: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:44.287834 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 17681
I20260806 07:53:44.306167 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17124
W20260806 07:53:44.306408 17816 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:44.250723 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:49551 (local address 127.20.202.254:34961)
0806 07:53:44.278508 (+ 27785us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:44.278517 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:44.279181 (+ 664us) server_negotiation.cc:408] Connection header received
0806 07:53:44.282643 (+ 3462us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:44.282652 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:44.282757 (+ 105us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:44.282870 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:44.305958 (+ 23088us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:49551: BlockingRecv error: recv got EOF from 127.20.202.193:49551 (error 108)
Metrics: {"server-negotiator.queue_time_us":27620,"thread_start_us":27783,"threads_started":1}
W20260806 07:53:44.307049 17815 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:44.250723 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:51921 (local address 127.20.202.253:41389)
0806 07:53:44.266532 (+ 15809us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:44.266578 (+ 46us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:44.279296 (+ 12718us) server_negotiation.cc:408] Connection header received
0806 07:53:44.282367 (+ 3071us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:44.282378 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:44.282511 (+ 133us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:44.282642 (+ 131us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:44.287798 (+ 5156us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:53:44.290863 (+ 3065us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:53:44.305930 (+ 15067us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:51921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":15605,"thread_start_us":16447,"threads_started":1}
W20260806 07:53:44.307601 17822 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:44.279479 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:44529 (local address 127.20.202.252:41411)
0806 07:53:44.287012 (+ 7533us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:44.287018 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:44.287037 (+ 19us) server_negotiation.cc:408] Connection header received
0806 07:53:44.287105 (+ 68us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:44.287110 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:44.287197 (+ 87us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:44.287318 (+ 121us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:44.306004 (+ 18686us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:44529: BlockingRecv error: recv got EOF from 127.20.202.193:44529 (error 108)
Metrics: {"server-negotiator.queue_time_us":7405,"thread_start_us":7215,"threads_started":1}
W20260806 07:53:44.309504 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17124: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:40449: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:44.309695 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 17124
I20260806 07:53:44.328085 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17268
W20260806 07:53:44.331169 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17268: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:34625: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:53:44.331343 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 17268
I20260806 07:53:44.349063 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17409
W20260806 07:53:44.351729 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 17409: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:34961: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:53:44.351897 17816 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:53:44.350284 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.20.202.254:34961)
0806 07:53:44.350497 (+ 213us) server_negotiation.cc:207] Beginning negotiation
0806 07:53:44.350504 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:53:44.350531 (+ 27us) server_negotiation.cc:408] Connection header received
0806 07:53:44.350615 (+ 84us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:53:44.350620 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:53:44.350712 (+ 92us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:53:44.350825 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:53:44.351630 (+ 805us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49174: BlockingRecv error: recv got EOF from 127.0.0.1:49174 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 07:53:44.351929 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 17409
I20260806 07:53:44.375161 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16766
I20260806 07:53:44.376685 16828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:44.385293 16844 master_runner.cc:187] Error getting master registration for 127.20.202.254:34961: Network error: Client connection negotiation failed: client connection to 127.20.202.254:34961: connect: Connection refused (error 111),
I20260806 07:53:44.385555 16844 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:53:44.507402 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 16766
I20260806 07:53:44.533053 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 16844
I20260806 07:53:44.534397 16907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:53:44.645141 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 16844
Shutting down hive metastore.
2026-08-06T07:53:45Z chronyd exiting
2026-08-06T07:53:45Z chronyd exiting
2026-08-06T07:53:45Z chronyd exiting
2026-08-06T07:53:45Z chronyd exiting
2026-08-06T07:53:45Z chronyd exiting
Aug 06 07:53:45 dist-test-slave-g350 krb5kdc[17834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:53:45 dist-test-slave-g350 krb5kdc[17834](info): set up 2 sockets
Aug 06 07:53:45 dist-test-slave-g350 krb5kdc[17834](info): commencing operation
krb5kdc: starting...
W20260806 07:53:47.131166 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.028s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-admin@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "test-user@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "joe-interloper@KRBTEST.COM".
Authenticating as principal slave/admin@KRBTEST.COM with password.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:53:47 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:53:47Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:47Z Disabled control of system clock
2026-08-06T07:53:47Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:47Z Disabled control of system clock
2026-08-06T07:53:47Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:47Z Disabled control of system clock
2026-08-06T07:53:47Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:47Z Disabled control of system clock
2026-08-06T07:53:47Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:53:47Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "hive/127.0.0.1@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:53:48: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:53:50 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002830, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:53:55 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002835, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:53:55 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002835, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.254@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:57.166949 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:57.527357 18037 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:57.527709 18037 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:57.527801 18037 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:57.536360 18037 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:57.536481 18037 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:57.536540 18037 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:57.536581 18037 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:57.547803 18037 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:57.549495 18037 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:57.551034 18037 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:57.560104 18042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:57.560462 18043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:57.561689 18045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:57.562312 18037 server_base.cc:1061] running on GCE node
I20260806 07:53:57.563222 18037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:57.565052 18037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:57.566437 18037 hybrid_clock.cc:648] HybridClock initialized: now 1786002837566230 us; error 171 us; skew 500 ppm
I20260806 07:53:57.566835 18037 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:57 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002837, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:57.570629 18037 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:53:57.572530 18037 webserver.cc:533] Webserver started at http://127.20.202.254:45083/ using document root <none> and password file <none>
I20260806 07:53:57.573321 18037 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:57.573465 18037 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:57.579243 18037 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 07:53:57.582358 18052 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:57.583819 18037 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:53:57.584013 18037 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "db950c71d6d14609b7f545c7d85763d4"
format_stamp: "Formatted at 2026-08-06 07:53:37 on dist-test-slave-g350"
I20260806 07:53:57.584769 18037 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:57.605299 18037 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:57.609860 18037 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:57.610153 18037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:57.631717 18037 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34961
I20260806 07:53:57.631753 18104 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34961 every 8 connection(s)
I20260806 07:53:57.633428 18037 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:53:57.638937 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18037
I20260806 07:53:57.639173 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:53:57.641474 18105 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:57.647040 18105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap starting.
I20260806 07:53:57.666831 18105 log.cc:826] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Log is configured to *not* fsync() on all Append() calls
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:57.678851 18105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:57.679522 18105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap complete.
I20260806 07:53:57.687189 18105 raft_consensus.cc:359] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:57.687947 18105 raft_consensus.cc:740] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db950c71d6d14609b7f545c7d85763d4, State: Initialized, Role: FOLLOWER
I20260806 07:53:57.688736 18105 consensus_queue.cc:260] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:57.689786 18110 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:57.690102 18110 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:57.691008 18105 sys_catalog.cc:565] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:57.703651 18121 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:57.703910 18121 catalog_manager.cc:1588] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: loading cluster ID for follower catalog manager: success
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:57.721673 18121 catalog_manager.cc:1610] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: acquiring CA information for follower catalog manager: success
I20260806 07:53:57.724828 18121 catalog_manager.cc:1638] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:53:57 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002837, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:53:57.737954 18037 master_runner.cc:187] Error getting master registration for 127.20.202.253:41389: Network error: Client connection negotiation failed: client connection to 127.20.202.253:41389: connect: Connection refused (error 111),
I20260806 07:53:57.741881 18037 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.253@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:57.789000 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:41389
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:58.155025 18127 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:58.155365 18127 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:58.155452 18127 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:58.164053 18127 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:58.164173 18127 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:58.164230 18127 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:58.164273 18127 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:58.175693 18127 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:58.177302 18127 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:58.178726 18127 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:58.187531 18133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:58.187498 18132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:58.188011 18135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:58.188253 18127 server_base.cc:1061] running on GCE node
I20260806 07:53:58.189142 18127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:58.190863 18127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:58.192099 18127 hybrid_clock.cc:648] HybridClock initialized: now 1786002838191958 us; error 115 us; skew 500 ppm
I20260806 07:53:58.192454 18127 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:58 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:58.195928 18127 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:53:58.197705 18127 webserver.cc:533] Webserver started at http://127.20.202.253:44325/ using document root <none> and password file <none>
I20260806 07:53:58.198500 18127 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:58.198657 18127 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:58.204178 18127 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:53:58.207481 18142 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:58.208868 18127 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:53:58.209069 18127 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "def6bb4233af46a287e75eed9c6b97cf"
format_stamp: "Formatted at 2026-08-06 07:53:38 on dist-test-slave-g350"
I20260806 07:53:58.209873 18127 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:58.234843 18127 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:58.239956 18127 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:58.240307 18127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:58.263265 18127 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:41389
I20260806 07:53:58.263298 18193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:41389 every 8 connection(s)
I20260806 07:53:58.265013 18127 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:53:58.265362 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18127
I20260806 07:53:58.265642 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:53:58.275449 18194 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:58.280624 18194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Bootstrap starting.
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:58.298657 18194 log.cc:826] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:58.308835 18194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:58.309537 18194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Bootstrap complete.
I20260806 07:53:58.316874 18194 raft_consensus.cc:359] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:58.317673 18194 raft_consensus.cc:740] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: def6bb4233af46a287e75eed9c6b97cf, State: Initialized, Role: FOLLOWER
I20260806 07:53:58.318547 18194 consensus_queue.cc:260] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:58.319526 18199 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:58.319897 18199 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:58.320566 18194 sys_catalog.cc:565] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: configured and running, proceeding with master startup.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:58.332541 18210 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:58.332723 18210 catalog_manager.cc:1588] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: loading cluster ID for follower catalog manager: success
I20260806 07:53:58.337105 18210 catalog_manager.cc:1610] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: acquiring CA information for follower catalog manager: success
I20260806 07:53:58.339964 18210 catalog_manager.cc:1638] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:53:58 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.252@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
I20260806 07:53:58.370817 18127 master_runner.cc:187] Error getting master registration for 127.20.202.252:41411: Network error: Client connection negotiation failed: client connection to 127.20.202.252:41411: connect: Connection refused (error 111),
I20260806 07:53:58.377946 18127 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:58.413854 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41411
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:53:58.797272 18218 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:53:58.797573 18218 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:53:58.797729 18218 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:53:58.806403 18218 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:53:58.806516 18218 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:53:58.806573 18218 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:53:58.806622 18218 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:53:58.818094 18218 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:41411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:53:58.819818 18218 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:53:58.821256 18218 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:53:58.830438 18223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:58.830490 18224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:53:58.831039 18226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:53:58.832295 18218 server_base.cc:1061] running on GCE node
I20260806 07:53:58.833159 18218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:53:58.835053 18218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:53:58.836316 18218 hybrid_clock.cc:648] HybridClock initialized: now 1786002838836155 us; error 126 us; skew 500 ppm
I20260806 07:53:58.836719 18218 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:53:58 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:53:58.841269 18218 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:53:58.843330 18218 webserver.cc:533] Webserver started at http://127.20.202.252:33929/ using document root <none> and password file <none>
I20260806 07:53:58.844202 18218 fs_manager.cc:362] Metadata directory not provided
I20260806 07:53:58.844379 18218 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:53:58.851102 18218 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:53:58.854769 18233 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:53:58.856786 18218 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:53:58.857052 18218 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "e122a8d66b7d4114adbc03c6be597e4f"
format_stamp: "Formatted at 2026-08-06 07:53:39 on dist-test-slave-g350"
I20260806 07:53:58.857956 18218 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:53:58.909972 18218 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:53:58.914299 18218 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:53:58.914644 18218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:53:58.936765 18284 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:41411 every 8 connection(s)
I20260806 07:53:58.936796 18218 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:41411
I20260806 07:53:58.938639 18218 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:53:58.941439 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18218
I20260806 07:53:58.941697 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:53:58.947912 18285 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:53:58.951604 18285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Bootstrap starting.
Aug 06 07:53:58 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:53:58 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:53:58.969933 18285 log.cc:826] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:53:58.980094 18285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:53:58.980656 18285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: Bootstrap complete.
Aug 06 07:53:58 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:53:58.987844 18285 raft_consensus.cc:359] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:58.988736 18285 raft_consensus.cc:740] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e122a8d66b7d4114adbc03c6be597e4f, State: Initialized, Role: FOLLOWER
I20260806 07:53:58.989498 18285 consensus_queue.cc:260] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:58.990691 18292 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:58.991113 18292 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:58.991580 18285 sys_catalog.cc:565] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:53:59.003245 18303 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:59.003361 18303 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: loading cluster ID for follower catalog manager: success
I20260806 07:53:59.007566 18303 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: acquiring CA information for follower catalog manager: success
I20260806 07:53:59.010962 18303 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:53:59 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:53:59.046088 18218 master_runner.cc:264] No leader master found from master e122a8d66b7d4114adbc03c6be597e4f
I20260806 07:53:59.046271 18218 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:53:59.563112 18306 raft_consensus.cc:493] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:53:59.563421 18306 raft_consensus.cc:515] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:59.565770 18306 leader_election.cc:290] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers def6bb4233af46a287e75eed9c6b97cf (127.20.202.253:41389), e122a8d66b7d4114adbc03c6be597e4f (127.20.202.252:41411)
I20260806 07:53:59.573729 18260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db950c71d6d14609b7f545c7d85763d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e122a8d66b7d4114adbc03c6be597e4f" is_pre_election: true
I20260806 07:53:59.573829 18169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db950c71d6d14609b7f545c7d85763d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "def6bb4233af46a287e75eed9c6b97cf" is_pre_election: true
I20260806 07:53:59.574242 18260 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db950c71d6d14609b7f545c7d85763d4 in term 1.
I20260806 07:53:59.574287 18169 raft_consensus.cc:2468] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db950c71d6d14609b7f545c7d85763d4 in term 1.
I20260806 07:53:59.574898 18054 leader_election.cc:304] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db950c71d6d14609b7f545c7d85763d4, def6bb4233af46a287e75eed9c6b97cf; no voters:
I20260806 07:53:59.575337 18306 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:53:59.575523 18306 raft_consensus.cc:493] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:53:59.575672 18306 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:53:59.579674 18306 raft_consensus.cc:515] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:59.580412 18306 leader_election.cc:290] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [CANDIDATE]: Term 2 election: Requested vote from peers def6bb4233af46a287e75eed9c6b97cf (127.20.202.253:41389), e122a8d66b7d4114adbc03c6be597e4f (127.20.202.252:41411)
I20260806 07:53:59.581043 18169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db950c71d6d14609b7f545c7d85763d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "def6bb4233af46a287e75eed9c6b97cf"
I20260806 07:53:59.581192 18260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db950c71d6d14609b7f545c7d85763d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e122a8d66b7d4114adbc03c6be597e4f"
I20260806 07:53:59.581286 18169 raft_consensus.cc:3060] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:53:59.581466 18260 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:53:59.584980 18169 raft_consensus.cc:2468] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db950c71d6d14609b7f545c7d85763d4 in term 2.
I20260806 07:53:59.585076 18260 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db950c71d6d14609b7f545c7d85763d4 in term 2.
I20260806 07:53:59.585481 18054 leader_election.cc:304] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db950c71d6d14609b7f545c7d85763d4, def6bb4233af46a287e75eed9c6b97cf; no voters:
I20260806 07:53:59.585844 18306 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:53:59.586297 18306 raft_consensus.cc:697] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 LEADER]: Becoming Leader. State: Replica: db950c71d6d14609b7f545c7d85763d4, State: Running, Role: LEADER
I20260806 07:53:59.586812 18306 consensus_queue.cc:237] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:53:59.590003 18311 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db950c71d6d14609b7f545c7d85763d4. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:59.590335 18311 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:59.590973 18313 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:53:59.593492 18313 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:53:59.594482 18313 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:53:59.594587 18313 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:53:59.595888 18313 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:53:59.596752 18313 catalog_manager.cc:6081] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Loaded TSK: 0
I20260806 07:53:59.597774 18313 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.193@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:53:59.725543 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:53:59.988515 18169 raft_consensus.cc:1275] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Refusing update from remote peer db950c71d6d14609b7f545c7d85763d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:53:59.989676 18311 consensus_queue.cc:1048] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:53:59.993975 18319 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader db950c71d6d14609b7f545c7d85763d4. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:59.994311 18319 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:59.995428 18306 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:59.995733 18306 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: LEADER
I20260806 07:53:59.997023 18319 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:53:59.997121 18260 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 2 FOLLOWER]: Refusing update from remote peer db950c71d6d14609b7f545c7d85763d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260806 07:53:59.997226 18319 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:53:59.997949 18311 consensus_queue.cc:1048] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:00.002090 18326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:00.002671 18326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:00.002945 18323 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader db950c71d6d14609b7f545c7d85763d4. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:00.003175 18323 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:54:00.086390 18318 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:00.086784 18318 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:00.086944 18318 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:00.096324 18318 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:00.096582 18318 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:00.107690 18318 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:00.109440 18318 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:00.110958 18318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:00.121449 18331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:00.121434 18332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:00.122041 18334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:00.122768 18318 server_base.cc:1061] running on GCE node
I20260806 07:54:00.123420 18318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:00.124750 18318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:00.126168 18318 hybrid_clock.cc:648] HybridClock initialized: now 1786002840126019 us; error 114 us; skew 500 ppm
I20260806 07:54:00.126529 18318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:00.130618 18318 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:54:00.132382 18318 webserver.cc:533] Webserver started at http://127.20.202.193:41995/ using document root <none> and password file <none>
I20260806 07:54:00.133183 18318 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:00.133328 18318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:00.139415 18318 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:54:00.142848 18341 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:00.144484 18318 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:54:00.144735 18318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:54:00.145608 18318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:00.180631 18318 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:00.188570 18318 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:00.188908 18318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:00.190168 18318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:00.191870 18318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:00.191994 18318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:00.192152 18318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:00.192255 18318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:00.229357 18318 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44089
I20260806 07:54:00.229404 18453 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44089 every 8 connection(s)
I20260806 07:54:00.231017 18318 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:00.237578 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18318
I20260806 07:54:00.237918 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:54:00.278713 18456 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:54:00.278744 18454 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:54:00.279170 18454 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:00.279175 18456 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:00.280175 18454 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:54:00.280230 18456 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:54:00.282773 18250 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:44089)
I20260806 07:54:00.282827 18159 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:44089)
I20260806 07:54:00.283463 18455 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:54:00.283604 18455 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:00.284198 18455 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:54:00.285936 18069 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:44089)
I20260806 07:54:00.287640 18069 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:39077
WARNING: no policy specified for kudu/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.194@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.194@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.194 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:54:00.395224 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:54:00.600811 18311 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "db950c71d6d14609b7f545c7d85763d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:00.601178 18311 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: LEADER
W20260806 07:54:00.755501 18468 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:00.755941 18468 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:00.756102 18468 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:00.764842 18468 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:00.765122 18468 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:54:00.776798 18468 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:00.778810 18468 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:00.780472 18468 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:00.792135 18476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:00.792256 18477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:00.792762 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:00.793780 18468 server_base.cc:1061] running on GCE node
I20260806 07:54:00.794471 18468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:00.795763 18468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:00.797016 18468 hybrid_clock.cc:648] HybridClock initialized: now 1786002840796876 us; error 100 us; skew 500 ppm
I20260806 07:54:00.797370 18468 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:00.801067 18468 init.cc:377] Logged in from keytab as kudu/127.20.202.194@KRBTEST.COM (short username kudu)
I20260806 07:54:00.802894 18468 webserver.cc:533] Webserver started at http://127.20.202.194:42471/ using document root <none> and password file <none>
I20260806 07:54:00.803692 18468 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:00.803857 18468 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:00.810065 18468 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:54:00.812913 18487 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:00.814479 18468 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:00.814704 18468 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "dda75ebc05424ff48d8dc725bf7468f9"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:54:00.815498 18468 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:00.836549 18468 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:00.840600 18468 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:00.840911 18468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:00.842069 18468 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:00.843806 18468 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:00.843937 18468 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:00.844084 18468 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:00.844189 18468 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:00.881491 18468 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:35171
I20260806 07:54:00.881537 18601 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:35171 every 8 connection(s)
I20260806 07:54:00.883176 18468 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:54:00.888687 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18468
I20260806 07:54:00.888980 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:54:00 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.194@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:54:00.937773 18603 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:54:00.938238 18603 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:00.939436 18603 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:54:00.940593 18604 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:54:00.940753 18604 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:00.941304 18604 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:54:00.941507 18069 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:35171)
I20260806 07:54:00.942656 18159 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:35171)
I20260806 07:54:00.942910 18069 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:40065
I20260806 07:54:00.943715 18602 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:54:00.943852 18602 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:00.944438 18602 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:54:00.945662 18250 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:35171)
WARNING: no policy specified for kudu/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "kudu/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.195@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.202.195@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.195 with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:54:01.050736 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:54:01.290576 18455 heartbeater.cc:499] Master 127.20.202.254:34961 was elected leader, sending a full tablet report...
W20260806 07:54:01.411166 18616 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:01.411473 18616 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:01.411562 18616 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:01.420104 18616 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:01.420557 18616 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:54:01.432277 18616 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:01.433993 18616 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:01.435559 18616 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:01.445955 18621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:01.446765 18624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:01.446736 18622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:01.447206 18616 server_base.cc:1061] running on GCE node
I20260806 07:54:01.447856 18616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:01.448988 18616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:01.452350 18616 hybrid_clock.cc:648] HybridClock initialized: now 1786002841452172 us; error 139 us; skew 500 ppm
I20260806 07:54:01.452718 18616 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:01 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:01.457033 18616 init.cc:377] Logged in from keytab as kudu/127.20.202.195@KRBTEST.COM (short username kudu)
I20260806 07:54:01.459090 18616 webserver.cc:533] Webserver started at http://127.20.202.195:44009/ using document root <none> and password file <none>
I20260806 07:54:01.459944 18616 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:01.460115 18616 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:01.466518 18616 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:54:01.469722 18631 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:01.471236 18616 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:54:01.471421 18616 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "3f3e95a8d2e44bab9356973be5491835"
format_stamp: "Formatted at 2026-08-06 07:53:41 on dist-test-slave-g350"
I20260806 07:54:01.472149 18616 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:01.524973 18616 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:01.529059 18616 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:01.529390 18616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:01.530534 18616 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:01.532258 18616 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:01.532387 18616 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:01.532519 18616 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:01.532608 18616 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:01.569800 18616 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:38757
I20260806 07:54:01.569859 18744 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:38757 every 8 connection(s)
I20260806 07:54:01.571398 18616 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:54:01.573263 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18616
I20260806 07:54:01.573463 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
Aug 06 07:54:01 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:54:01 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:54:01 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.195@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:54:01.622560 18746 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:54:01.622596 18745 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:54:01.623050 18745 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:01.623061 18746 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:01.623301 18747 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:54:01.623447 18747 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:01.624269 18747 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:54:01.624270 18745 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:54:01.624392 18746 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:54:01.626173 18069 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:38757)
I20260806 07:54:01.626276 18159 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:38757)
I20260806 07:54:01.626449 18250 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:38757)
I20260806 07:54:01.627305 18069 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:45321
I20260806 07:54:01.635370 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:54:01.644275 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:54:01.644436 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18037
I20260806 07:54:01.645354 18100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:01.760192 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18037
I20260806 07:54:01.784870 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--webserver_interface=127.20.202.254
--webserver_port=45083
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:54:01.946601 18603 heartbeater.cc:499] Master 127.20.202.254:34961 was elected leader, sending a full tablet report...
W20260806 07:54:01.948750 18603 heartbeater.cc:646] Failed to heartbeat to 127.20.202.254:34961 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.202.254:34961: connect: Connection refused (error 111)
W20260806 07:54:02.154248 18756 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:02.154585 18756 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:02.154764 18756 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:02.163273 18756 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:02.163389 18756 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:02.163508 18756 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:02.163643 18756 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:02.175724 18756 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=45083
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:02.177539 18756 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:02.179025 18756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:02.187915 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:02.188329 18762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:02.188530 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:02.189281 18756 server_base.cc:1061] running on GCE node
I20260806 07:54:02.190160 18756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:02.192363 18756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:02.193655 18756 hybrid_clock.cc:648] HybridClock initialized: now 1786002842193467 us; error 119 us; skew 500 ppm
I20260806 07:54:02.194075 18756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002842, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:02.198016 18756 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:54:02.199779 18756 webserver.cc:533] Webserver started at http://127.20.202.254:45083/ using document root <none> and password file <none>
I20260806 07:54:02.200570 18756 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:02.200706 18756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:02.206632 18756 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 07:54:02.209697 18772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:02.211185 18756 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:54:02.211395 18756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "db950c71d6d14609b7f545c7d85763d4"
format_stamp: "Formatted at 2026-08-06 07:53:37 on dist-test-slave-g350"
I20260806 07:54:02.212217 18756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:02.239223 18756 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:02.243609 18756 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:02.243927 18756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:02.269193 18756 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34961
I20260806 07:54:02.269235 18824 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34961 every 8 connection(s)
I20260806 07:54:02.271435 18756 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:02.276697 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18756
I20260806 07:54:02.277062 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18318
I20260806 07:54:02.280001 18825 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:02.283653 18825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap starting.
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:54:02.300477 18449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:02.306025 18825 log.cc:826] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:02.315461 18825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:02.316035 18825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: Bootstrap complete.
I20260806 07:54:02.322751 18825 raft_consensus.cc:359] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:54:02.323499 18825 raft_consensus.cc:740] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: db950c71d6d14609b7f545c7d85763d4, State: Initialized, Role: FOLLOWER
I20260806 07:54:02.324268 18825 consensus_queue.cc:260] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:54:02.325187 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:02.325512 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:02.326372 18825 sys_catalog.cc:565] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:02.341143 18844 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:54:02.341364 18844 catalog_manager.cc:1588] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: loading cluster ID for follower catalog manager: success
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002842, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:54:02.346057 18844 catalog_manager.cc:1610] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: acquiring CA information for follower catalog manager: success
I20260806 07:54:02.349032 18844 catalog_manager.cc:1638] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:02.387701 18756 master_runner.cc:264] No leader master found from master db950c71d6d14609b7f545c7d85763d4
I20260806 07:54:02.387867 18756 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:02.420984 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18318
I20260806 07:54:02.448067 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44089
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--webserver_port=41995
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:54:02.448621 18830 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:02.294281 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:49121 (local address 127.20.202.254:34961)
0806 07:54:02.295626 (+ 1345us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:02.295668 (+ 42us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:02.295730 (+ 62us) server_negotiation.cc:408] Connection header received
0806 07:54:02.298096 (+ 2366us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:02.298133 (+ 37us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:02.298787 (+ 654us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:02.299412 (+ 625us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:02.301070 (+ 1658us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:54:02.303585 (+ 2515us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:54:02.421145 (+117560us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:54:02.425861 (+ 4716us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:54:02.446951 (+ 21090us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:49121: BlockingRecv error: recv got EOF from 127.20.202.193:49121 (error 108)
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":1322,"threads_started":1}
I20260806 07:54:02.629472 18746 heartbeater.cc:499] Master 127.20.202.254:34961 was elected leader, sending a full tablet report...
I20260806 07:54:02.640765 18789 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f3e95a8d2e44bab9356973be5491835" instance_seqno: 1786002841561136) as {username='kudu', principal='kudu/127.20.202.195@KRBTEST.COM'} at 127.20.202.195:58177; Asking this server to re-register.
I20260806 07:54:02.641834 18746 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:02.643844 18789 ts_manager.cc:194] Registered new tserver with Master: 3f3e95a8d2e44bab9356973be5491835 (127.20.202.195:38757)
W20260806 07:54:02.821710 18850 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:02.822064 18850 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:02.822217 18850 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:02.830955 18850 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:02.831156 18850 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:02.842736 18850 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41995
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:02.844446 18850 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:02.846055 18850 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:02.856463 18857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:02.856730 18859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:02.856406 18856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:02.858458 18850 server_base.cc:1061] running on GCE node
I20260806 07:54:02.859088 18850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:02.860306 18850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:02.861682 18850 hybrid_clock.cc:648] HybridClock initialized: now 1786002842861533 us; error 100 us; skew 500 ppm
I20260806 07:54:02.861981 18850 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002842, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:02.865839 18850 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:54:02.867515 18850 webserver.cc:533] Webserver started at http://127.20.202.193:41995/ using document root <none> and password file <none>
I20260806 07:54:02.868273 18850 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:02.868402 18850 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:02.874287 18850 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:54:02.877308 18866 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:02.877790 18867 raft_consensus.cc:493] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db950c71d6d14609b7f545c7d85763d4)
I20260806 07:54:02.878022 18867 raft_consensus.cc:515] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:54:02.878891 18850 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:02.879071 18850 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:54:02.879807 18867 leader_election.cc:290] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db950c71d6d14609b7f545c7d85763d4 (127.20.202.254:34961), e122a8d66b7d4114adbc03c6be597e4f (127.20.202.252:41411)
I20260806 07:54:02.879944 18850 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:02.889091 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db950c71d6d14609b7f545c7d85763d4" is_pre_election: true
I20260806 07:54:02.889669 18799 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate def6bb4233af46a287e75eed9c6b97cf in term 2.
I20260806 07:54:02.889914 18260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e122a8d66b7d4114adbc03c6be597e4f" is_pre_election: true
I20260806 07:54:02.890635 18145 leader_election.cc:304] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db950c71d6d14609b7f545c7d85763d4, def6bb4233af46a287e75eed9c6b97cf; no voters:
I20260806 07:54:02.891105 18867 raft_consensus.cc:2804] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 07:54:02.891252 18867 raft_consensus.cc:493] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Starting leader election (detected failure of leader db950c71d6d14609b7f545c7d85763d4)
I20260806 07:54:02.891381 18867 raft_consensus.cc:3060] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:54:02.893942 18867 raft_consensus.cc:515] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:54:02.894636 18867 leader_election.cc:290] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 3 election: Requested vote from peers db950c71d6d14609b7f545c7d85763d4 (127.20.202.254:34961), e122a8d66b7d4114adbc03c6be597e4f (127.20.202.252:41411)
I20260806 07:54:02.895052 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db950c71d6d14609b7f545c7d85763d4"
I20260806 07:54:02.895197 18260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "def6bb4233af46a287e75eed9c6b97cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e122a8d66b7d4114adbc03c6be597e4f"
I20260806 07:54:02.895284 18799 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:54:02.900657 18799 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate def6bb4233af46a287e75eed9c6b97cf in term 3.
I20260806 07:54:02.901255 18145 leader_election.cc:304] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db950c71d6d14609b7f545c7d85763d4, def6bb4233af46a287e75eed9c6b97cf; no voters: e122a8d66b7d4114adbc03c6be597e4f
I20260806 07:54:02.901731 18867 raft_consensus.cc:2804] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 3 FOLLOWER]: Leader election won for term 3
I20260806 07:54:02.902202 18867 raft_consensus.cc:697] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [term 3 LEADER]: Becoming Leader. State: Replica: def6bb4233af46a287e75eed9c6b97cf, State: Running, Role: LEADER
I20260806 07:54:02.902629 18850 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:02.902673 18867 consensus_queue.cc:237] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:54:02.904701 18874 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:02.905272 18874 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:02.906071 18876 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:02.907760 18850 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:02.908052 18850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:02.908098 18876 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:02.909016 18876 catalog_manager.cc:1295] Loaded cluster ID: c6b9d6b3fda249e798c4610ae57f74ef
I20260806 07:54:02.909129 18876 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:02.909138 18850 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:02.910429 18876 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:02.910995 18850 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:02.911136 18850 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:02.911257 18850 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:02.911345 18850 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:02.911396 18876 catalog_manager.cc:6081] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf: Loaded TSK: 0
I20260806 07:54:02.912386 18876 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:02.954569 18850 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44089
I20260806 07:54:02.954775 18986 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44089 every 8 connection(s)
I20260806 07:54:02.956704 18850 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:02.959105 18603 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:54:02.959312 18603 heartbeater.cc:499] Master 127.20.202.254:34961 was elected leader, sending a full tablet report...
I20260806 07:54:02.960340 18789 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dda75ebc05424ff48d8dc725bf7468f9" instance_seqno: 1786002840872776) as {username='kudu', principal='kudu/127.20.202.194@KRBTEST.COM'} at 127.20.202.194:52145; Asking this server to re-register.
I20260806 07:54:02.961083 18603 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:02.962772 18789 ts_manager.cc:194] Registered new tserver with Master: dda75ebc05424ff48d8dc725bf7468f9 (127.20.202.194:35171)
I20260806 07:54:02.965237 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 18850
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002842, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002842, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:54:02 dist-test-slave-g350 krb5kdc[17834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002842, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:54:02.997136 18988 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34961
I20260806 07:54:02.997575 18988 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:02.998723 18988 heartbeater.cc:507] Master 127.20.202.254:34961 requested a full tablet report, sending...
I20260806 07:54:03.000952 18789 ts_manager.cc:194] Registered new tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:44089)
I20260806 07:54:03.004899 18991 heartbeater.cc:344] Connected to a master server at 127.20.202.253:41389
I20260806 07:54:03.005067 18991 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:03.005656 18991 heartbeater.cc:507] Master 127.20.202.253:41389 requested a full tablet report, sending...
I20260806 07:54:03.007097 18159 ts_manager.cc:194] Re-registered known tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:44089)
I20260806 07:54:03.009001 18159 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:59969
I20260806 07:54:03.011330 18987 heartbeater.cc:344] Connected to a master server at 127.20.202.252:41411
I20260806 07:54:03.011461 18987 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:03.012042 18987 heartbeater.cc:507] Master 127.20.202.252:41411 requested a full tablet report, sending...
I20260806 07:54:03.014178 18250 ts_manager.cc:194] Re-registered known tserver with Master: ef23bc519c3d43b8bca0f64ea7550de1 (127.20.202.193:44089)
I20260806 07:54:03.041033 18996 raft_consensus.cc:493] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db950c71d6d14609b7f545c7d85763d4)
I20260806 07:54:03.041297 18996 raft_consensus.cc:515] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } }
I20260806 07:54:03.043382 18996 leader_election.cc:290] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db950c71d6d14609b7f545c7d85763d4 (127.20.202.254:34961), def6bb4233af46a287e75eed9c6b97cf (127.20.202.253:41389)
I20260806 07:54:03.044147 18169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e122a8d66b7d4114adbc03c6be597e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "def6bb4233af46a287e75eed9c6b97cf" is_pre_election: true
I20260806 07:54:03.050974 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e122a8d66b7d4114adbc03c6be597e4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db950c71d6d14609b7f545c7d85763d4" is_pre_election: true
I20260806 07:54:03.051391 18799 raft_consensus.cc:2393] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e122a8d66b7d4114adbc03c6be597e4f in current term 3: Already voted for candidate def6bb4233af46a287e75eed9c6b97cf in this term.
I20260806 07:54:03.052146 18236 leader_election.cc:304] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e122a8d66b7d4114adbc03c6be597e4f; no voters: db950c71d6d14609b7f545c7d85763d4, def6bb4233af46a287e75eed9c6b97cf
I20260806 07:54:03.052568 18996 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 2 FOLLOWER]: Advancing to term 3
I20260806 07:54:03.055328 18996 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260806 07:54:03.139843 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18850
W20260806 07:54:03.141247 19001 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:03.137175 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.20.202.193:44089)
0806 07:54:03.138139 (+ 964us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:03.138146 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:03.138176 (+ 30us) server_negotiation.cc:408] Connection header received
0806 07:54:03.138244 (+ 68us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:03.138252 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:03.138545 (+ 293us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:03.138722 (+ 177us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:03.139733 (+ 1011us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34460: BlockingRecv error: recv got EOF from 127.0.0.1:34460 (error 108)
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":637,"threads_started":1}
W20260806 07:54:03.142386 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18850: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:44089: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:54:03.142566 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18850
W20260806 07:54:03.142576 19002 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:03.140598 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.20.202.193:44089)
0806 07:54:03.141258 (+ 660us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:03.141266 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:03.141291 (+ 25us) server_negotiation.cc:408] Connection header received
0806 07:54:03.141346 (+ 55us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:03.141351 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:03.141421 (+ 70us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:03.141547 (+ 126us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:03.142352 (+ 805us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34466: BlockingRecv error: recv got EOF from 127.0.0.1:34466 (error 108)
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":355,"threads_started":1}
I20260806 07:54:03.161705 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44089
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--webserver_port=41995
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:54:03.281100 18799 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [term 3 FOLLOWER]: Refusing update from remote peer def6bb4233af46a287e75eed9c6b97cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:54:03.282202 18874 consensus_queue.cc:1048] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:03.285265 19004 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:03.285583 19004 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:03.287577 18867 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:03.287842 18867 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:03.288352 18260 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [term 3 FOLLOWER]: Refusing update from remote peer def6bb4233af46a287e75eed9c6b97cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260806 07:54:03.289147 18867 consensus_queue.cc:1048] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:03.300313 19004 sys_catalog.cc:455] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:03.300949 19004 sys_catalog.cc:458] T 00000000000000000000000000000000 P db950c71d6d14609b7f545c7d85763d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:03.302886 18996 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader def6bb4233af46a287e75eed9c6b97cf. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:03.303205 18996 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:03.302944 19011 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:03.303388 19011 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122a8d66b7d4114adbc03c6be597e4f [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:54:03.525229 19003 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:03.525547 19003 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:03.525700 19003 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:54:03.525768 19003 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:03.534366 19003 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:03.534590 19003 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:03.545356 19003 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41995
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:03.547042 19003 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:03.548414 19003 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:03.558980 19017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:03.559003 19016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:03.559186 19019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:03.559976 19003 server_base.cc:1061] running on GCE node
I20260806 07:54:03.560649 19003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:03.561870 19003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:03.563254 19003 hybrid_clock.cc:648] HybridClock initialized: now 1786002843563060 us; error 142 us; skew 500 ppm
I20260806 07:54:03.563599 19003 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:03 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:03.567440 19003 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:54:03.569257 19003 webserver.cc:533] Webserver started at http://127.20.202.193:41995/ using document root <none> and password file <none>
I20260806 07:54:03.570187 19003 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:03.570310 19003 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:03.576287 19003 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:54:03.579370 19027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:03.580698 19003 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 07:54:03.580883 19003 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:54:03.581676 19003 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:03.599524 19003 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:03.603312 19003 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:03.603615 19003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:03.604609 19003 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:03.606344 19003 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:03.606451 19003 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:03.606601 19003 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:03.606674 19003 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:54:03.607153 19003 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:54:03.607244 19003 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:54:03.607329 19003 fault_injection.cc:59] injecting fault for kudu (pid 19003): FLAGS_fault_before_start (process will exit)
I20260806 07:54:03.632795 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44089
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--webserver_port=41995
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:54:03.806785 19007 sys_catalog.cc:455] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "def6bb4233af46a287e75eed9c6b97cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db950c71d6d14609b7f545c7d85763d4" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34961 } } peers { permanent_uuid: "def6bb4233af46a287e75eed9c6b97cf" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 41389 } } peers { permanent_uuid: "e122a8d66b7d4114adbc03c6be597e4f" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 41411 } } }
I20260806 07:54:03.807119 19007 sys_catalog.cc:458] T 00000000000000000000000000000000 P def6bb4233af46a287e75eed9c6b97cf [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:03.951412 18604 heartbeater.cc:499] Master 127.20.202.253:41389 was elected leader, sending a full tablet report...
W20260806 07:54:03.993332 19035 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:03.993713 19035 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:03.993825 19035 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:04.002343 19035 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:04.002615 19035 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:04.013705 19035 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:40265,127.20.202.213:41221,127.20.202.214:38301,127.20.202.215:38471,127.20.202.216:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=41995
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34961,127.20.202.253:41389,127.20.202.252:41411
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:04.015470 19035 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:04.016845 19035 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:04.026914 19044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:04.027108 19043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:04.027567 19046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:04.028610 19035 server_base.cc:1061] running on GCE node
I20260806 07:54:04.029227 19035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:04.030498 19035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:04.031767 19035 hybrid_clock.cc:648] HybridClock initialized: now 1786002844031614 us; error 117 us; skew 500 ppm
I20260806 07:54:04.032150 19035 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:54:04 dist-test-slave-g350 krb5kdc[17834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002844, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:54:04.036216 19035 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:54:04.037979 19035 webserver.cc:533] Webserver started at http://127.20.202.193:41995/ using document root <none> and password file <none>
I20260806 07:54:04.038762 19035 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:04.038920 19035 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:04.045059 19035 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 07:54:04.048662 19053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:04.050711 19035 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 07:54:04.050918 19035 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ef23bc519c3d43b8bca0f64ea7550de1"
format_stamp: "Formatted at 2026-08-06 07:53:40 on dist-test-slave-g350"
I20260806 07:54:04.051771 19035 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:04.075676 19035 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:04.080309 19035 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:04.080577 19035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:04.081723 19035 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:04.083623 19035 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:04.083721 19035 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:04.083806 19035 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:04.083866 19035 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:04.122216 19035 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44089
I20260806 07:54:04.122308 19168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44089 every 8 connection(s)
I20260806 07:54:04.123924 19035 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestBasicOperation_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:04.124346 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19035
I20260806 07:54:04.124733 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19035
W20260806 07:54:04.145457 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19035: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:44089: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:54:04.145816 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19035
W20260806 07:54:04.163820 19174 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:04.130172 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:58533 (local address 127.20.202.254:34961)
0806 07:54:04.138094 (+ 7922us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:04.138103 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:04.142601 (+ 4498us) server_negotiation.cc:408] Connection header received
0806 07:54:04.143559 (+ 958us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:04.143569 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:04.143706 (+ 137us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:04.143826 (+ 120us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:04.163552 (+ 19726us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:58533: BlockingRecv error: recv got EOF from 127.20.202.193:58533 (error 108)
Metrics: {"server-negotiator.queue_time_us":7758,"thread_start_us":7610,"threads_started":1}
I20260806 07:54:04.163815 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18468
W20260806 07:54:04.164808 19175 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:04.131003 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:47557 (local address 127.20.202.252:41411)
0806 07:54:04.145162 (+ 14159us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:04.145168 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:04.145189 (+ 21us) server_negotiation.cc:408] Connection header received
0806 07:54:04.145251 (+ 62us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:04.145260 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:04.145348 (+ 88us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:04.149833 (+ 4485us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:04.163486 (+ 13653us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:47557: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":14020,"thread_start_us":18899,"threads_started":1}
W20260806 07:54:04.166996 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18468: Not authorized: Client connection negotiation failed: client connection to 127.20.202.194:35171: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:54:04.167156 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18468
I20260806 07:54:04.185237 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18616
W20260806 07:54:04.188278 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18616: Not authorized: Client connection negotiation failed: client connection to 127.20.202.195:38757: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:54:04.188484 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18616
I20260806 07:54:04.206840 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18756
W20260806 07:54:04.209105 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18756: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:34961: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:54:04.209264 19174 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:04.207824 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.20.202.254:34961)
0806 07:54:04.208011 (+ 187us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:04.208018 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:04.208041 (+ 23us) server_negotiation.cc:408] Connection header received
0806 07:54:04.208108 (+ 67us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:04.208113 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:04.208184 (+ 71us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:04.208278 (+ 94us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:04.208940 (+ 662us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41844: BlockingRecv error: recv got EOF from 127.0.0.1:41844 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 07:54:04.209287 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18756
I20260806 07:54:04.233434 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18127
I20260806 07:54:04.234871 18189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:04.354231 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18127
I20260806 07:54:04.377314 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 18218
I20260806 07:54:04.378692 18280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:04.489645 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 18218
Shutting down hive metastore.
2026-08-06T07:54:04Z chronyd exiting
2026-08-06T07:54:04Z chronyd exiting
2026-08-06T07:54:04Z chronyd exiting
2026-08-06T07:54:04Z chronyd exiting
2026-08-06T07:54:04Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestBasicOperation/11 (48037 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/0
I20260806 07:54:04.965001 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39879
--webserver_interface=127.20.202.254
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373 with env {}
W20260806 07:54:05.318895 19187 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:05.319234 19187 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:05.319367 19187 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:05.327857 19187 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:05.327961 19187 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:05.328063 19187 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:05.328133 19187 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:05.338385 19187 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:05.340056 19187 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:05.342008 19187 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:05.350479 19192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:05.350610 19193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:05.351634 19195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:05.352443 19187 server_base.cc:1061] running on GCE node
I20260806 07:54:05.353195 19187 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:05.353391 19187 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:05.353494 19187 hybrid_clock.cc:648] HybridClock initialized: now 1786002845353471 us; error 0 us; skew 500 ppm
I20260806 07:54:05.353884 19187 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:05.356679 19187 webserver.cc:533] Webserver started at http://127.20.202.254:38757/ using document root <none> and password file <none>
I20260806 07:54:05.357442 19187 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:05.357580 19187 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:05.358036 19187 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:05.360384 19187 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "c820dff5db0647178dda6037036ea208"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:05.361186 19187 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "c820dff5db0647178dda6037036ea208"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:05.366729 19187 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 07:54:05.370661 19200 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:05.371963 19187 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:54:05.372162 19187 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c820dff5db0647178dda6037036ea208"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:05.372386 19187 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:05.397425 19187 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:05.398465 19187 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:05.398777 19187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:05.420190 19187 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39879
I20260806 07:54:05.420213 19251 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39879 every 8 connection(s)
I20260806 07:54:05.421841 19187 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:05.426074 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19187
I20260806 07:54:05.426358 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:54:05.426542 19252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:05.428001 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:33559
--webserver_interface=127.20.202.253
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373 with env {}
I20260806 07:54:05.436507 19252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:05.460832 19252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:05.462431 19202 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:33559: connect: Connection refused (error 111)
W20260806 07:54:05.464529 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:33559: Network error: Client connection negotiation failed: client connection to 127.20.202.253:33559: connect: Connection refused (error 111)
I20260806 07:54:05.513974 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } attempt: 1
W20260806 07:54:05.516376 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:33559: Network error: Client connection negotiation failed: client connection to 127.20.202.253:33559: connect: Connection refused (error 111)
I20260806 07:54:05.584702 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } attempt: 2
W20260806 07:54:05.586920 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:33559: Network error: Client connection negotiation failed: client connection to 127.20.202.253:33559: connect: Connection refused (error 111)
I20260806 07:54:05.678162 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } attempt: 3
W20260806 07:54:05.680430 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:33559: Network error: Client connection negotiation failed: client connection to 127.20.202.253:33559: connect: Connection refused (error 111)
W20260806 07:54:05.790686 19254 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:05.791065 19254 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:05.791218 19254 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:05.799846 19254 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:05.799959 19254 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:05.800067 19254 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:05.800163 19254 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:05.810669 19254 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:33559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:05.812518 19254 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:05.814507 19254 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:05.823050 19263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:05.823048 19264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:05.823470 19266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:05.823700 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } attempt: 4
I20260806 07:54:05.824105 19254 server_base.cc:1061] running on GCE node
I20260806 07:54:05.825198 19254 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:05.825429 19254 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:05.825541 19254 hybrid_clock.cc:648] HybridClock initialized: now 1786002845825516 us; error 0 us; skew 500 ppm
W20260806 07:54:05.825911 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:33559: Network error: Client connection negotiation failed: client connection to 127.20.202.253:33559: connect: Connection refused (error 111)
I20260806 07:54:05.826068 19254 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:05.829278 19254 webserver.cc:533] Webserver started at http://127.20.202.253:38755/ using document root <none> and password file <none>
I20260806 07:54:05.830240 19254 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:05.830348 19254 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:05.830772 19254 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:05.833518 19254 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "3d369bee58624f6388864bbe1d59dbcc"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:05.834522 19254 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "3d369bee58624f6388864bbe1d59dbcc"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:05.840382 19254 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:54:05.844106 19271 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:05.845491 19254 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:54:05.845777 19254 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "3d369bee58624f6388864bbe1d59dbcc"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:05.845980 19254 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:05.886297 19254 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:05.887216 19254 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:05.887468 19254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:05.909024 19254 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:33559
I20260806 07:54:05.909121 19324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:33559 every 8 connection(s)
I20260806 07:54:05.910710 19254 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:54:05.915253 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19254
I20260806 07:54:05.915465 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:54:05.915518 19325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:05.916841 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:34373
--webserver_interface=127.20.202.252
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373 with env {}
I20260806 07:54:05.924633 19325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:05.942950 19325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:05.954522 19325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:05.955848 19275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
W20260806 07:54:05.957681 19325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.007069 19325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 1
W20260806 07:54:06.009411 19325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.077678 19325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 2
W20260806 07:54:06.080178 19325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.125216 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } attempt: 5
I20260806 07:54:06.132522 19252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:06.134534 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.171459 19325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 3
W20260806 07:54:06.173821 19325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.185775 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 1
W20260806 07:54:06.188088 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.256373 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 2
W20260806 07:54:06.258628 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
W20260806 07:54:06.285786 19328 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:06.286126 19328 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:06.286206 19328 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:06.294855 19328 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:06.294958 19328 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:06.295017 19328 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:06.295065 19328 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:06.305410 19328 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:34373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:06.307152 19328 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:06.308949 19328 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:54:06.317127 19325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 4
W20260806 07:54:06.317883 19335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:06.319065 19336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:06.319361 19325 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
W20260806 07:54:06.319921 19338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:06.321185 19328 server_base.cc:1061] running on GCE node
I20260806 07:54:06.322091 19328 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:06.322247 19328 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:06.322315 19328 hybrid_clock.cc:648] HybridClock initialized: now 1786002846322297 us; error 0 us; skew 500 ppm
I20260806 07:54:06.322616 19328 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:06.325350 19328 webserver.cc:533] Webserver started at http://127.20.202.252:45519/ using document root <none> and password file <none>
I20260806 07:54:06.326211 19328 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:06.326311 19328 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:06.326666 19328 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:06.329041 19328 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "d147df210c814e0294d0eebcdf7b8b53"
format_stamp: "Formatted at 2026-08-06 07:54:06 on dist-test-slave-g350"
I20260806 07:54:06.329931 19328 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "d147df210c814e0294d0eebcdf7b8b53"
format_stamp: "Formatted at 2026-08-06 07:54:06 on dist-test-slave-g350"
I20260806 07:54:06.335606 19328 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:54:06.339105 19343 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:06.340492 19328 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 07:54:06.340662 19328 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "d147df210c814e0294d0eebcdf7b8b53"
format_stamp: "Formatted at 2026-08-06 07:54:06 on dist-test-slave-g350"
I20260806 07:54:06.340862 19328 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:06.352686 19328 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:06.353547 19328 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:06.353878 19328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:06.364890 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 3
W20260806 07:54:06.367319 19252 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34373: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34373: connect: Connection refused (error 111)
I20260806 07:54:06.374545 19395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:34373 every 8 connection(s)
I20260806 07:54:06.374567 19328 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:34373
I20260806 07:54:06.376260 19328 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:54:06.380695 19396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:06.384723 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19328
I20260806 07:54:06.384927 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:54:06.386792 19396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:06.401948 19396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:06.408475 19396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:06.418876 19396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: Bootstrap starting.
I20260806 07:54:06.422224 19396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:06.423311 19396 log.cc:826] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:06.426638 19396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: No bootstrap required, opened a new log
I20260806 07:54:06.431857 19396 raft_consensus.cc:359] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.432145 19396 raft_consensus.cc:385] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:06.432296 19396 raft_consensus.cc:740] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d147df210c814e0294d0eebcdf7b8b53, State: Initialized, Role: FOLLOWER
I20260806 07:54:06.432925 19396 consensus_queue.cc:260] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.433873 19403 sys_catalog.cc:455] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.434171 19403 sys_catalog.cc:458] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.435077 19396 sys_catalog.cc:565] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:06.445190 19414 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:06.445312 19414 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:06.443519 19328 master_runner.cc:187] Error getting master registration for 127.20.202.254:39879: OK, instance_id { permanent_uuid: "c820dff5db0647178dda6037036ea208" instance_seqno: 1786002845414311 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:06.448674 19328 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:06.544569 19252 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 4
I20260806 07:54:06.555599 19252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Bootstrap starting.
I20260806 07:54:06.558959 19252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:06.560081 19252 log.cc:826] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:06.562736 19252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: No bootstrap required, opened a new log
I20260806 07:54:06.567876 19252 raft_consensus.cc:359] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.568152 19252 raft_consensus.cc:385] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:06.568315 19252 raft_consensus.cc:740] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c820dff5db0647178dda6037036ea208, State: Initialized, Role: FOLLOWER
I20260806 07:54:06.568961 19252 consensus_queue.cc:260] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.569967 19417 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.570274 19417 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.571092 19252 sys_catalog.cc:565] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:06.582835 19428 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:06.583065 19428 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:06.581646 19187 master_runner.cc:187] Error getting master registration for 127.20.202.253:33559: OK, instance_id { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" instance_seqno: 1786002845903260 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:06.588474 19187 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:54:06.605967 19349 tablet.cc:2406] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:06.618579 19325 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } attempt: 5
I20260806 07:54:06.633349 19325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: Bootstrap starting.
I20260806 07:54:06.637832 19325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:06.639351 19325 log.cc:826] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:06.642542 19325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: No bootstrap required, opened a new log
I20260806 07:54:06.650396 19325 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.650750 19325 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:06.650934 19325 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d369bee58624f6388864bbe1d59dbcc, State: Initialized, Role: FOLLOWER
W20260806 07:54:06.651645 19206 tablet.cc:2406] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:06.652717 19325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.653777 19431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.654124 19431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.655691 19325 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:06.668764 19431 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:54:06.669008 19431 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.671823 19371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d369bee58624f6388864bbe1d59dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d147df210c814e0294d0eebcdf7b8b53" is_pre_election: true
I20260806 07:54:06.672520 19371 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d369bee58624f6388864bbe1d59dbcc in term 0.
I20260806 07:54:06.672798 19227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d369bee58624f6388864bbe1d59dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c820dff5db0647178dda6037036ea208" is_pre_election: true
I20260806 07:54:06.673297 19227 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d369bee58624f6388864bbe1d59dbcc in term 0.
I20260806 07:54:06.673303 19275 leader_election.cc:304] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d369bee58624f6388864bbe1d59dbcc, d147df210c814e0294d0eebcdf7b8b53; no voters:
I20260806 07:54:06.673960 19431 leader_election.cc:290] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c820dff5db0647178dda6037036ea208 (127.20.202.254:39879), d147df210c814e0294d0eebcdf7b8b53 (127.20.202.252:34373)
I20260806 07:54:06.674279 19431 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:54:06.674481 19431 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:54:06.674659 19431 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:06.674041 19254 master_runner.cc:264] No leader master found from master 3d369bee58624f6388864bbe1d59dbcc
I20260806 07:54:06.674829 19254 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:06.678385 19431 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.679452 19227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d369bee58624f6388864bbe1d59dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c820dff5db0647178dda6037036ea208"
I20260806 07:54:06.679658 19227 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:06.683012 19227 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d369bee58624f6388864bbe1d59dbcc in term 1.
I20260806 07:54:06.685019 19274 leader_election.cc:304] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d369bee58624f6388864bbe1d59dbcc, c820dff5db0647178dda6037036ea208; no voters:
I20260806 07:54:06.685614 19431 leader_election.cc:290] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [CANDIDATE]: Term 1 election: Requested vote from peers c820dff5db0647178dda6037036ea208 (127.20.202.254:39879), d147df210c814e0294d0eebcdf7b8b53 (127.20.202.252:34373)
I20260806 07:54:06.685989 19431 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:54:06.686187 19431 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 1 LEADER]: Becoming Leader. State: Replica: 3d369bee58624f6388864bbe1d59dbcc, State: Running, Role: LEADER
I20260806 07:54:06.686786 19431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } }
I20260806 07:54:06.690246 19371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d369bee58624f6388864bbe1d59dbcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d147df210c814e0294d0eebcdf7b8b53"
I20260806 07:54:06.690480 19371 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:06.694424 19371 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d369bee58624f6388864bbe1d59dbcc in term 1.
W20260806 07:54:06.704455 19443 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:06.704617 19443 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:06.705713 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d369bee58624f6388864bbe1d59dbcc. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.706208 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:06.715538 19447 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:06.718240 19447 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:06.733319 19371 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 1 FOLLOWER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:06.738067 19445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:06.746768 19227 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:06.748045 19445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:06.771178 19417 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d369bee58624f6388864bbe1d59dbcc. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.771448 19417 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.771919 19448 mvcc.cc:204] Tried to move back new op lower bound from 7315467660208316416 to 7315467660035280896. Current Snapshot: MvccSnapshot[applied={T|T < 7315467660208316416}]
I20260806 07:54:06.776051 19403 sys_catalog.cc:455] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d369bee58624f6388864bbe1d59dbcc. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.776337 19403 sys_catalog.cc:458] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.778242 19431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.778548 19431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:06.780911 19417 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.781149 19417 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.782385 19447 catalog_manager.cc:1383] Generated new cluster ID: 758edbc3638c4cb79b1dc9d7ab1e53ea
I20260806 07:54:06.782505 19447 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:06.786048 19455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.786459 19455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:06.790201 19403 sys_catalog.cc:455] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } }
I20260806 07:54:06.790407 19403 sys_catalog.cc:458] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:06.794818 19449 mvcc.cc:204] Tried to move back new op lower bound from 7315467660208316416 to 7315467660035280896. Current Snapshot: MvccSnapshot[applied={T|T < 7315467660208316416 or (T in {7315467660208316416})}]
I20260806 07:54:06.825634 19447 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:54:06.827543 19447 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:06.859776 19447 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: Generated new TSK 0
I20260806 07:54:06.861181 19447 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:06.899897 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--time_source=system_unsync with env {}
W20260806 07:54:07.339811 19458 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:07.340129 19458 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:07.340211 19458 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:07.348664 19458 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:07.348834 19458 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:07.359491 19458 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:07.361191 19458 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:07.363149 19458 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:07.373807 19464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:07.373922 19466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:07.374541 19463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:07.375272 19458 server_base.cc:1061] running on GCE node
I20260806 07:54:07.375890 19458 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:07.376075 19458 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:07.376190 19458 hybrid_clock.cc:648] HybridClock initialized: now 1786002847376168 us; error 0 us; skew 500 ppm
I20260806 07:54:07.376531 19458 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:07.379066 19458 webserver.cc:533] Webserver started at http://127.20.202.193:38471/ using document root <none> and password file <none>
I20260806 07:54:07.379907 19458 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:07.380036 19458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:07.380442 19458 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:07.383090 19458 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:07.383945 19458 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:07.389744 19458 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:54:07.393487 19471 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:07.395174 19458 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:07.395419 19458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:07.395701 19458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:07.447347 19414 catalog_manager.cc:1295] Loaded cluster ID: 758edbc3638c4cb79b1dc9d7ab1e53ea
I20260806 07:54:07.447479 19414 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: loading cluster ID for follower catalog manager: success
I20260806 07:54:07.450362 19414 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: acquiring CA information for follower catalog manager: success
I20260806 07:54:07.452302 19414 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:07.454154 19458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:07.455250 19458 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:07.455586 19458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:07.456763 19458 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:07.458724 19458 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:07.458850 19458 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:07.459017 19458 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:07.459123 19458 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:07.497503 19458 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43005
I20260806 07:54:07.497561 19585 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43005 every 8 connection(s)
I20260806 07:54:07.499118 19458 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:07.504547 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19458
I20260806 07:54:07.504824 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:54:07.529448 19586 heartbeater.cc:344] Connected to a master server at 127.20.202.252:34373
I20260806 07:54:07.529492 19587 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39879
I20260806 07:54:07.529708 19588 heartbeater.cc:344] Connected to a master server at 127.20.202.253:33559
I20260806 07:54:07.529973 19588 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:07.529982 19587 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:07.530978 19588 heartbeater.cc:507] Master 127.20.202.253:33559 requested a full tablet report, sending...
I20260806 07:54:07.531013 19587 heartbeater.cc:507] Master 127.20.202.254:39879 requested a full tablet report, sending...
I20260806 07:54:07.531898 19586 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:07.532450 19586 heartbeater.cc:507] Master 127.20.202.252:34373 requested a full tablet report, sending...
I20260806 07:54:07.533435 19289 ts_manager.cc:194] Registered new tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:07.533588 19216 ts_manager.cc:194] Registered new tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:07.534313 19361 ts_manager.cc:194] Registered new tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:07.535550 19289 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:51967
I20260806 07:54:07.539129 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:07.542114 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:46815
--webserver_interface=127.20.202.251
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815 with env {}
I20260806 07:54:07.585013 19428 catalog_manager.cc:1295] Loaded cluster ID: 758edbc3638c4cb79b1dc9d7ab1e53ea
I20260806 07:54:07.585153 19428 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: loading cluster ID for follower catalog manager: success
I20260806 07:54:07.587879 19428 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: acquiring CA information for follower catalog manager: success
I20260806 07:54:07.589705 19428 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:54:07.901682 19596 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:07.902055 19596 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:07.902202 19596 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:07.910826 19596 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:07.910918 19596 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:07.911026 19596 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:07.911094 19596 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:07.921412 19596 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:46815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:07.923233 19596 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:07.925076 19596 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:07.933024 19602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:07.933059 19601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:07.933759 19604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:07.934713 19596 server_base.cc:1061] running on GCE node
I20260806 07:54:07.935448 19596 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:07.935644 19596 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:07.935755 19596 hybrid_clock.cc:648] HybridClock initialized: now 1786002847935733 us; error 0 us; skew 500 ppm
I20260806 07:54:07.936088 19596 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:07.939265 19596 webserver.cc:533] Webserver started at http://127.20.202.251:45805/ using document root <none> and password file <none>
I20260806 07:54:07.940289 19596 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:07.940420 19596 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:07.940872 19596 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:07.944376 19596 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "0a37af00868047b1bb4007700bf63662"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:07.945375 19596 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "0a37af00868047b1bb4007700bf63662"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:07.952347 19596 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:54:07.956163 19609 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:07.957805 19596 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:07.958010 19596 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "0a37af00868047b1bb4007700bf63662"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:07.958249 19596 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:08.019879 19596 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:08.020960 19596 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:08.021306 19596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:08.044847 19596 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:46815
I20260806 07:54:08.044891 19660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:46815 every 8 connection(s)
I20260806 07:54:08.046837 19596 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:08.051457 19661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:08.054998 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19596
I20260806 07:54:08.058014 19661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:08.071195 19661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:08.077926 19661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:08.084137 19661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:08.093463 19661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Bootstrap starting.
I20260806 07:54:08.096764 19661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:08.097893 19661 log.cc:826] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:08.100436 19661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: No bootstrap required, opened a new log
I20260806 07:54:08.106606 19661 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } }
I20260806 07:54:08.107019 19661 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:08.107154 19661 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a37af00868047b1bb4007700bf63662, State: Initialized, Role: FOLLOWER
I20260806 07:54:08.107834 19661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } }
I20260806 07:54:08.108754 19670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } } }
I20260806 07:54:08.109025 19670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:08.109863 19661 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:08.118840 19681 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:08.118983 19681 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:08.125077 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19458
I20260806 07:54:08.127749 19596 master_runner.cc:429] Detected that this master 0a37af00868047b1bb4007700bf63662 is joining an existing cluster
I20260806 07:54:08.127864 19596 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:54:08.128913 19596 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:54:08.129343 19596 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:54:08.129647 19596 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: stopping tablet replica
I20260806 07:54:08.132251 19596 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:46815
I20260806 07:54:08.135360 19581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:08.176271 19289 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:46815: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0a37af00868047b1bb4007700bf63662"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 46815
}
attrs {
promote: true
}
}
dest_uuid: "3d369bee58624f6388864bbe1d59dbcc"
cas_config_opid_index: -1
I20260806 07:54:08.178311 19289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } }
I20260806 07:54:08.182034 19227 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:54:08.182530 19272 consensus_peers.cc:597] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc -> Peer 0a37af00868047b1bb4007700bf63662 (127.20.202.251:46815): Couldn't send request to peer 0a37af00868047b1bb4007700bf63662. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:54:08.182824 19445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:08.184300 19371 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 1 FOLLOWER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:08.185067 19445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:08.199298 19226 raft_consensus.cc:2955] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0a37af00868047b1bb4007700bf63662 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.199932 19371 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0a37af00868047b1bb4007700bf63662 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.198486 19698 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0a37af00868047b1bb4007700bf63662 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.210676 19700 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.211174 19700 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:08.214109 19698 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:46815
I20260806 07:54:08.215337 19596 master.cc:562] Master@127.20.202.251:46815 shutting down...
I20260806 07:54:08.217484 19699 sys_catalog.cc:455] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.217834 19699 sys_catalog.cc:458] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:08.247131 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.247418 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:08.248185 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.248390 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:08.248770 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.248955 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:08.256281 19596 master.cc:584] Master@127.20.202.251:46815 shutdown complete.
I20260806 07:54:08.256481 19596 master_runner.cc:306] Clearing existing system tablet
I20260806 07:54:08.257092 19596 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:08.257256 19596 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:08.260088 19596 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 07:54:08.261814 19706 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:08.262189 19596 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:08.262488 19596 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 07:54:08.262631 19596 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "0a37af00868047b1bb4007700bf63662"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:08.266147 19596 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:54:08.271340 19596 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:54:08.271530 19596 log.cc:1199] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:54:08.272167 19596 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Deleting consensus metadata
I20260806 07:54:08.272506 19596 master_runner.cc:316] Copying system tablet from 127.20.202.253:33559
I20260806 07:54:08.279126 19596 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.253:33559
I20260806 07:54:08.287878 19309 tablet_copy_service.cc:140] P 3d369bee58624f6388864bbe1d59dbcc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0a37af00868047b1bb4007700bf63662 ({username='slave'} at 127.0.0.1:47620)
I20260806 07:54:08.288328 19309 tablet_copy_service.cc:161] P 3d369bee58624f6388864bbe1d59dbcc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0a37af00868047b1bb4007700bf63662 at {username='slave'} at 127.0.0.1:47620: session id = 0a37af00868047b1bb4007700bf63662-00000000000000000000000000000000
I20260806 07:54:08.292793 19309 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:54:08.293694 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19458
I20260806 07:54:08.295934 19596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:54:08.305104 19596 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: tablet copy: Starting download of 0 data blocks...
I20260806 07:54:08.305614 19596 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: tablet copy: Starting download of 1 WAL segments...
I20260806 07:54:08.310405 19596 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:54:08.316717 19309 tablet_copy_service.cc:342] P 3d369bee58624f6388864bbe1d59dbcc: Request end of tablet copy session 0a37af00868047b1bb4007700bf63662-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:47620
I20260806 07:54:08.317143 19309 tablet_copy_service.cc:434] P 3d369bee58624f6388864bbe1d59dbcc: ending tablet copy session 0a37af00868047b1bb4007700bf63662-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0a37af00868047b1bb4007700bf63662
I20260806 07:54:08.322456 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43005
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--webserver_port=38471
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815 with env {}
I20260806 07:54:08.330142 19596 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:08.333262 19713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:08.333948 19714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:08.341055 19716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:08.341815 19596 server_base.cc:1061] running on GCE node
I20260806 07:54:08.342175 19596 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:08.342283 19596 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:08.342365 19596 hybrid_clock.cc:648] HybridClock initialized: now 1786002848342342 us; error 0 us; skew 500 ppm
I20260806 07:54:08.342638 19596 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:08.344183 19596 webserver.cc:533] Webserver started at http://127.20.202.251:43971/ using document root <none> and password file <none>
I20260806 07:54:08.344624 19596 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:08.344745 19596 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:08.348094 19596 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 07:54:08.350332 19722 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:08.350795 19596 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:54:08.350925 19596 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "0a37af00868047b1bb4007700bf63662"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:08.351200 19596 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:08.406430 19596 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:08.407449 19596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:08.464931 19596 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:46815
I20260806 07:54:08.464970 19774 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:46815 every 8 connection(s)
I20260806 07:54:08.466162 19596 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:08.470901 19775 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:08.474761 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Bootstrap starting.
I20260806 07:54:08.493109 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:08.493757 19775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: Bootstrap complete.
I20260806 07:54:08.494941 19775 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } }
I20260806 07:54:08.495211 19775 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a37af00868047b1bb4007700bf63662, State: Initialized, Role: LEARNER
I20260806 07:54:08.495462 19775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } }
I20260806 07:54:08.496356 19779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.496587 19775 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:08.496815 19779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:08.506032 19790 catalog_manager.cc:1295] Loaded cluster ID: 758edbc3638c4cb79b1dc9d7ab1e53ea
I20260806 07:54:08.506150 19790 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: loading cluster ID for follower catalog manager: success
I20260806 07:54:08.510031 19790 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: acquiring CA information for follower catalog manager: success
I20260806 07:54:08.512568 19790 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:08.598153 19750 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:54:08.599005 19779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d369bee58624f6388864bbe1d59dbcc. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:08.599318 19779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:54:08.759200 19718 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:08.759539 19718 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:08.759706 19718 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:08.769172 19718 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:08.769340 19718 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:08.780048 19718 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43005
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38471
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:08.781721 19718 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:08.783545 19718 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:08.794035 19797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:08.794137 19796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:08.794106 19799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:08.795029 19718 server_base.cc:1061] running on GCE node
I20260806 07:54:08.795678 19718 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:08.795903 19718 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:08.796015 19718 hybrid_clock.cc:648] HybridClock initialized: now 1786002848795993 us; error 0 us; skew 500 ppm
I20260806 07:54:08.796345 19718 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:08.798707 19718 webserver.cc:533] Webserver started at http://127.20.202.193:38471/ using document root <none> and password file <none>
I20260806 07:54:08.799628 19718 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:08.799785 19718 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:08.805465 19718 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:54:08.808228 19804 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:08.809772 19718 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:54:08.809962 19718 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:08.810712 19718 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:08.836303 19718 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:08.837359 19718 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:08.837754 19718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:08.838862 19718 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:08.840835 19718 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:08.840941 19718 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:08.841096 19718 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:08.841169 19718 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:08.876507 19718 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43005
I20260806 07:54:08.876574 19918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43005 every 8 connection(s)
I20260806 07:54:08.878446 19718 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:08.881170 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19718
I20260806 07:54:08.921922 19919 heartbeater.cc:344] Connected to a master server at 127.20.202.251:46815
I20260806 07:54:08.922398 19919 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:08.923326 19919 heartbeater.cc:507] Master 127.20.202.251:46815 requested a full tablet report, sending...
I20260806 07:54:08.926362 19740 ts_manager.cc:194] Registered new tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:08.926956 19920 heartbeater.cc:344] Connected to a master server at 127.20.202.252:34373
I20260806 07:54:08.927150 19920 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:08.927835 19920 heartbeater.cc:507] Master 127.20.202.252:34373 requested a full tablet report, sending...
I20260806 07:54:08.929294 19361 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:08.930019 19921 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39879
I20260806 07:54:08.930154 19921 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:08.930605 19921 heartbeater.cc:507] Master 127.20.202.254:39879 requested a full tablet report, sending...
I20260806 07:54:08.932189 19216 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:08.935261 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:08.942965 19922 heartbeater.cc:344] Connected to a master server at 127.20.202.253:33559
I20260806 07:54:08.943111 19922 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:08.943656 19922 heartbeater.cc:507] Master 127.20.202.253:33559 requested a full tablet report, sending...
I20260806 07:54:08.944890 19288 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:08.945966 19288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:33593
I20260806 07:54:08.957330 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:08.957567 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19187
I20260806 07:54:08.958666 19247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:09.042109 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:09.042550 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:09.044073 19935 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc: attempting to promote NON_VOTER 0a37af00868047b1bb4007700bf63662 to VOTER
I20260806 07:54:09.045090 19935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } }
I20260806 07:54:09.048045 19371 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 1 FOLLOWER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:54:09.048167 19750 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 1 LEARNER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:54:09.048772 19937 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:09.049255 19935 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:09.053452 19935 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0a37af00868047b1bb4007700bf63662 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.054263 19750 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0a37af00868047b1bb4007700bf63662 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.057579 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.057871 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:09.057771 19937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.057968 19935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.058034 19937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:09.058691 19779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.058904 19779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a37af00868047b1bb4007700bf63662 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:09.058820 19371 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0a37af00868047b1bb4007700bf63662 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.058236 19935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:09.063253 19939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.063565 19939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d147df210c814e0294d0eebcdf7b8b53 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:09.076532 19226 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Refusing update from remote peer 3d369bee58624f6388864bbe1d59dbcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:54:09.077224 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19187
I20260806 07:54:09.077328 19935 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:54:09.101630 19274 connection.cc:573] client connection to 127.20.202.254:39879 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:09.102304 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39879
--webserver_interface=127.20.202.254
--webserver_port=38757
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815 with env {}
W20260806 07:54:09.103757 19274 consensus_peers.cc:597] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc -> Peer c820dff5db0647178dda6037036ea208 (127.20.202.254:39879): Couldn't send request to peer c820dff5db0647178dda6037036ea208. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:39879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:54:09.476827 19942 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:09.477285 19942 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:09.477414 19942 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:09.486616 19942 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:09.486722 19942 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:09.486783 19942 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:09.486830 19942 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:09.497582 19942 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:39879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=38757
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:09.499526 19942 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:09.501727 19942 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:09.510841 19947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:09.511294 19948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:09.512027 19950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:09.512208 19942 server_base.cc:1061] running on GCE node
I20260806 07:54:09.513025 19942 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:09.513227 19942 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:09.513324 19942 hybrid_clock.cc:648] HybridClock initialized: now 1786002849513300 us; error 0 us; skew 500 ppm
I20260806 07:54:09.513761 19942 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:09.516782 19942 webserver.cc:533] Webserver started at http://127.20.202.254:38757/ using document root <none> and password file <none>
I20260806 07:54:09.517683 19942 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:09.517832 19942 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:09.524149 19942 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:54:09.527245 19955 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:09.528596 19942 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:54:09.528779 19942 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c820dff5db0647178dda6037036ea208"
format_stamp: "Formatted at 2026-08-06 07:54:05 on dist-test-slave-g350"
I20260806 07:54:09.529589 19942 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:09.549906 19942 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:09.550957 19942 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:09.551271 19942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:09.573278 19942 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:39879
I20260806 07:54:09.573320 20006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:39879 every 8 connection(s)
I20260806 07:54:09.575057 19942 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:09.582345 20007 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:09.582624 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 19942
I20260806 07:54:09.582963 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19718
I20260806 07:54:09.587016 20007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Bootstrap starting.
I20260806 07:54:09.592927 19914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:09.606889 20007 log.cc:826] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:09.616902 20007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:09.617527 20007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: Bootstrap complete.
I20260806 07:54:09.624408 20007 raft_consensus.cc:359] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } }
I20260806 07:54:09.625180 20007 raft_consensus.cc:740] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c820dff5db0647178dda6037036ea208, State: Initialized, Role: FOLLOWER
I20260806 07:54:09.625968 20007 consensus_queue.cc:260] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } }
I20260806 07:54:09.626876 20013 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: true } } }
I20260806 07:54:09.627182 20013 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:09.628000 20007 sys_catalog.cc:565] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:09.648495 20024 catalog_manager.cc:1295] Loaded cluster ID: 758edbc3638c4cb79b1dc9d7ab1e53ea
I20260806 07:54:09.648705 20024 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: loading cluster ID for follower catalog manager: success
I20260806 07:54:09.653507 20024 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: acquiring CA information for follower catalog manager: success
I20260806 07:54:09.656488 20024 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:09.710965 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19718
I20260806 07:54:09.711577 19982 raft_consensus.cc:2955] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0a37af00868047b1bb4007700bf63662 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.717453 20033 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.718015 20033 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:09.718428 20013 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d369bee58624f6388864bbe1d59dbcc. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:09.718616 20013 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820dff5db0647178dda6037036ea208 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:09.738251 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43005
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--webserver_port=38471
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815 with env {}
W20260806 07:54:10.118170 20034 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:10.118510 20034 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:10.118664 20034 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:10.127794 20034 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:10.128012 20034 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:10.138912 20034 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43005
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38471
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:10.140632 20034 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:10.142545 20034 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:10.152652 20040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:10.152706 20039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:10.153831 20042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:10.154330 20034 server_base.cc:1061] running on GCE node
I20260806 07:54:10.155074 20034 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:10.155287 20034 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:10.155375 20034 hybrid_clock.cc:648] HybridClock initialized: now 1786002850155356 us; error 0 us; skew 500 ppm
I20260806 07:54:10.155715 20034 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:10.158298 20034 webserver.cc:533] Webserver started at http://127.20.202.193:38471/ using document root <none> and password file <none>
I20260806 07:54:10.159106 20034 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:10.159199 20034 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:10.165120 20034 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:54:10.168211 20048 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:10.169803 20034 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:10.170060 20034 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:10.170826 20034 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:10.197901 20034 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:10.198971 20034 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:10.199330 20034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:10.200410 20034 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:10.202250 20034 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:10.202353 20034 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:10.202500 20034 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:10.202574 20034 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:10.230362 20127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d369bee58624f6388864bbe1d59dbcc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c820dff5db0647178dda6037036ea208" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 39879 } } peers { permanent_uuid: "3d369bee58624f6388864bbe1d59dbcc" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 33559 } } peers { permanent_uuid: "d147df210c814e0294d0eebcdf7b8b53" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34373 } } peers { permanent_uuid: "0a37af00868047b1bb4007700bf63662" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 46815 } attrs { promote: false } } }
I20260806 07:54:10.230624 20127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d369bee58624f6388864bbe1d59dbcc [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:10.238982 20034 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43005
I20260806 07:54:10.239032 20163 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43005 every 8 connection(s)
I20260806 07:54:10.240602 20034 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:10.248880 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20034
I20260806 07:54:10.249322 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20034
I20260806 07:54:10.291561 20167 heartbeater.cc:344] Connected to a master server at 127.20.202.253:33559
I20260806 07:54:10.291955 20167 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:10.293267 20167 heartbeater.cc:507] Master 127.20.202.253:33559 requested a full tablet report, sending...
I20260806 07:54:10.295529 19288 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:10.296218 20166 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39879
I20260806 07:54:10.296391 20166 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:10.296660 19288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:35257
I20260806 07:54:10.296909 20166 heartbeater.cc:507] Master 127.20.202.254:39879 requested a full tablet report, sending...
I20260806 07:54:10.298326 20164 heartbeater.cc:344] Connected to a master server at 127.20.202.251:46815
I20260806 07:54:10.298462 20164 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:10.298862 20159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:10.298947 20164 heartbeater.cc:507] Master 127.20.202.251:46815 requested a full tablet report, sending...
I20260806 07:54:10.299048 19972 ts_manager.cc:194] Registered new tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:10.300348 19740 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:10.394991 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20034
I20260806 07:54:10.414836 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43005
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--webserver_port=38471
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--fault_before_start=1.0 with env {}
W20260806 07:54:10.416137 20172 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:10.245150 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:39127 (local address 127.20.202.252:34373)
0806 07:54:10.291713 (+ 46563us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:10.291720 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:10.291740 (+ 20us) server_negotiation.cc:408] Connection header received
0806 07:54:10.291802 (+ 62us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:10.291807 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:10.291899 (+ 92us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:10.292006 (+ 107us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:10.293219 (+ 1213us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:54:10.294609 (+ 1390us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:54:10.396150 (+101541us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:54:10.396799 (+ 649us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:54:10.413815 (+ 17016us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:39127: BlockingRecv error: recv got EOF from 127.20.202.193:39127 (error 108)
Metrics: {"server-negotiator.queue_time_us":46412,"thread_start_us":46249,"threads_started":1}
W20260806 07:54:10.762768 20180 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:10.763149 20180 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:10.763316 20180 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:54:10.763412 20180 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:10.772869 20180 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:10.773195 20180 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:10.784082 20180 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43005
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38471
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:10.786082 20180 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:10.788116 20180 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:10.799117 20186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:10.799288 20185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:10.799849 20188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:10.800622 20180 server_base.cc:1061] running on GCE node
I20260806 07:54:10.801244 20180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:10.801421 20180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:10.801542 20180 hybrid_clock.cc:648] HybridClock initialized: now 1786002850801485 us; error 0 us; skew 500 ppm
I20260806 07:54:10.801893 20180 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:10.804198 20180 webserver.cc:533] Webserver started at http://127.20.202.193:38471/ using document root <none> and password file <none>
I20260806 07:54:10.805017 20180 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:10.805127 20180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:10.811141 20180 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:54:10.814116 20193 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:10.815714 20180 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:54:10.815968 20180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:10.816816 20180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:10.851585 20180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:10.852849 20180 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:10.853252 20180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:10.854547 20180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:10.856875 20180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:10.857029 20180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:10.857197 20180 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:10.857304 20180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:54:10.857779 20180 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:54:10.857920 20180 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:54:10.858027 20180 fault_injection.cc:59] injecting fault for kudu (pid 20180): FLAGS_fault_before_start (process will exit)
I20260806 07:54:10.876777 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43005
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373
--webserver_port=38471
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815 with env {}
W20260806 07:54:11.239897 20201 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:11.240239 20201 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:11.240376 20201 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:11.248987 20201 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:11.249224 20201 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:11.265128 20201 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43005
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=38471
--tserver_master_addrs=127.20.202.254:39879,127.20.202.253:33559,127.20.202.252:34373,127.20.202.251:46815
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:11.267474 20201 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:11.269513 20201 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:11.280592 20207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:11.280556 20206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:11.281701 20209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:11.282179 20201 server_base.cc:1061] running on GCE node
I20260806 07:54:11.282788 20201 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:11.282948 20201 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:11.283080 20201 hybrid_clock.cc:648] HybridClock initialized: now 1786002851283053 us; error 0 us; skew 500 ppm
I20260806 07:54:11.283368 20201 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:11.286154 20201 webserver.cc:533] Webserver started at http://127.20.202.193:38471/ using document root <none> and password file <none>
I20260806 07:54:11.287340 20201 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:11.287496 20201 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:11.293747 20201 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:54:11.296634 20214 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:11.298202 20201 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:11.298393 20201 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "e6a1dfc5cc9b4265a08914e09eaabb3e"
format_stamp: "Formatted at 2026-08-06 07:54:07 on dist-test-slave-g350"
I20260806 07:54:11.299118 20201 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:11.325563 20201 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:11.326656 20201 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:11.326987 20201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:11.328051 20201 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:11.329941 20201 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:11.330051 20201 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:11.330204 20201 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:11.330279 20201 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:11.366292 20201 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43005
I20260806 07:54:11.366330 20326 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43005 every 8 connection(s)
I20260806 07:54:11.368172 20201 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:11.369547 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20201
I20260806 07:54:11.369944 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20201
I20260806 07:54:11.416917 20328 heartbeater.cc:344] Connected to a master server at 127.20.202.252:34373
I20260806 07:54:11.417454 20328 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:11.417542 20327 heartbeater.cc:344] Connected to a master server at 127.20.202.251:46815
I20260806 07:54:11.417713 20327 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:11.418422 20329 heartbeater.cc:344] Connected to a master server at 127.20.202.254:39879
I20260806 07:54:11.418619 20329 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:11.418916 20327 heartbeater.cc:507] Master 127.20.202.251:46815 requested a full tablet report, sending...
I20260806 07:54:11.419018 20338 heartbeater.cc:344] Connected to a master server at 127.20.202.253:33559
I20260806 07:54:11.419241 20329 heartbeater.cc:507] Master 127.20.202.254:39879 requested a full tablet report, sending...
I20260806 07:54:11.419296 20328 heartbeater.cc:507] Master 127.20.202.252:34373 requested a full tablet report, sending...
I20260806 07:54:11.419353 20322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:11.419242 20338 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:11.421103 19740 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:11.421100 19361 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:11.422189 19972 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:11.510597 20338 heartbeater.cc:507] Master 127.20.202.253:33559 requested a full tablet report, sending...
I20260806 07:54:11.511283 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20201
I20260806 07:54:11.512957 19288 ts_manager.cc:194] Re-registered known tserver with Master: e6a1dfc5cc9b4265a08914e09eaabb3e (127.20.202.193:43005)
I20260806 07:54:11.513952 19288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:38125
W20260806 07:54:11.530584 19272 connection.cc:573] server connection from 127.20.202.193:38125 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:11.531297 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19942
I20260806 07:54:11.538241 20002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:11.644528 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19942
I20260806 07:54:11.668093 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19254
I20260806 07:54:11.669305 19320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:11.782274 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19254
I20260806 07:54:11.805879 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19328
I20260806 07:54:11.807130 19391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:11.917479 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19328
I20260806 07:54:11.940512 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 19596
I20260806 07:54:11.941884 19770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:12.063025 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 19596
[ OK ] ExternalMiniClusterTest.TestAddMaster/0 (7135 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/1
2026-08-06T07:54:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:12Z Disabled control of system clock
I20260806 07:54:12.141647 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43141
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575 with env {}
W20260806 07:54:12.499223 20353 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:12.499552 20353 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:12.499687 20353 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:12.508282 20353 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:12.508381 20353 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:12.508437 20353 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:12.508486 20353 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:12.518785 20353 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:12.520498 20353 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:12.522385 20353 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:12.531023 20359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:12.531109 20358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:12.532086 20361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:12.532724 20353 server_base.cc:1061] running on GCE node
I20260806 07:54:12.533499 20353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:12.535027 20353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:12.536244 20353 hybrid_clock.cc:648] HybridClock initialized: now 1786002852536178 us; error 60 us; skew 500 ppm
I20260806 07:54:12.536631 20353 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:12.539031 20353 webserver.cc:533] Webserver started at http://127.20.202.254:41095/ using document root <none> and password file <none>
I20260806 07:54:12.539810 20353 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:12.539909 20353 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:12.540277 20353 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:12.542814 20353 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "1c8fecf6baec490293b4d8f2e7365894"
format_stamp: "Formatted at 2026-08-06 07:54:12 on dist-test-slave-g350"
I20260806 07:54:12.543677 20353 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "1c8fecf6baec490293b4d8f2e7365894"
format_stamp: "Formatted at 2026-08-06 07:54:12 on dist-test-slave-g350"
I20260806 07:54:12.549127 20353 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 07:54:12.552718 20367 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:12.554183 20353 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:54:12.554384 20353 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "1c8fecf6baec490293b4d8f2e7365894"
format_stamp: "Formatted at 2026-08-06 07:54:12 on dist-test-slave-g350"
I20260806 07:54:12.554601 20353 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:12.588665 20353 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:12.589758 20353 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:12.590097 20353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:12.612280 20353 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43141
I20260806 07:54:12.612299 20418 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43141 every 8 connection(s)
I20260806 07:54:12.613947 20353 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:12.618532 20419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:12.622319 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20353
I20260806 07:54:12.622558 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:54:12.623584 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:36167
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575 with env {}
I20260806 07:54:12.624630 20419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:12.641861 20419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:12.643059 20368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:36167: connect: Connection refused (error 111)
W20260806 07:54:12.644806 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:36167: Network error: Client connection negotiation failed: client connection to 127.20.202.253:36167: connect: Connection refused (error 111)
I20260806 07:54:12.694204 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } attempt: 1
W20260806 07:54:12.696342 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:36167: Network error: Client connection negotiation failed: client connection to 127.20.202.253:36167: connect: Connection refused (error 111)
I20260806 07:54:12.764623 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } attempt: 2
W20260806 07:54:12.766772 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:36167: Network error: Client connection negotiation failed: client connection to 127.20.202.253:36167: connect: Connection refused (error 111)
I20260806 07:54:12.858103 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } attempt: 3
W20260806 07:54:12.860334 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:36167: Network error: Client connection negotiation failed: client connection to 127.20.202.253:36167: connect: Connection refused (error 111)
W20260806 07:54:12.993927 20423 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:12.994318 20423 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:12.994467 20423 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:54:13.003657 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } attempt: 4
W20260806 07:54:13.004170 20423 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:13.004297 20423 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:13.004388 20423 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:13.004462 20423 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:54:13.006246 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:36167: Network error: Client connection negotiation failed: client connection to 127.20.202.253:36167: connect: Connection refused (error 111)
I20260806 07:54:13.015679 20423 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:13.017463 20423 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:13.019510 20423 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:13.028363 20431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:13.028523 20430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:13.029839 20433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:13.029791 20423 server_base.cc:1061] running on GCE node
I20260806 07:54:13.030714 20423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:13.032152 20423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:13.033401 20423 hybrid_clock.cc:648] HybridClock initialized: now 1786002853033347 us; error 53 us; skew 500 ppm
I20260806 07:54:13.033840 20423 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:13.036326 20423 webserver.cc:533] Webserver started at http://127.20.202.253:33619/ using document root <none> and password file <none>
I20260806 07:54:13.037106 20423 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:13.037246 20423 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:13.037660 20423 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:13.040176 20423 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "60aa77b18ef04d748526dcf23325819b"
format_stamp: "Formatted at 2026-08-06 07:54:13 on dist-test-slave-g350"
I20260806 07:54:13.041060 20423 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "60aa77b18ef04d748526dcf23325819b"
format_stamp: "Formatted at 2026-08-06 07:54:13 on dist-test-slave-g350"
I20260806 07:54:13.046723 20423 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 07:54:13.050781 20439 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:13.052404 20423 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:54:13.052598 20423 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "60aa77b18ef04d748526dcf23325819b"
format_stamp: "Formatted at 2026-08-06 07:54:13 on dist-test-slave-g350"
I20260806 07:54:13.052832 20423 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:13.070982 20423 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:13.072023 20423 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:13.072345 20423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:13.094997 20423 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:36167
I20260806 07:54:13.095021 20491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:36167 every 8 connection(s)
I20260806 07:54:13.096769 20423 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:54:13.097707 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20423
I20260806 07:54:13.097916 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:54:13.099113 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33575
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575 with env {}
I20260806 07:54:13.101699 20492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:13.107969 20492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:13.123428 20492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:13.132665 20492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:13.133836 20441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
W20260806 07:54:13.135574 20492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.184928 20492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 1
W20260806 07:54:13.187311 20492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.255565 20492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 2
W20260806 07:54:13.258217 20492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.305611 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } attempt: 5
I20260806 07:54:13.313203 20419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:13.315420 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.349505 20492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 3
W20260806 07:54:13.352028 20492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.366662 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 1
W20260806 07:54:13.368932 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.437186 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 2
W20260806 07:54:13.439500 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
W20260806 07:54:13.465111 20494 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:13.465481 20494 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:13.465636 20494 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:13.474119 20494 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:13.474237 20494 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:13.474351 20494 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:13.474436 20494 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:13.484589 20494 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:13.486332 20494 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:13.488188 20494 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:54:13.495260 20492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 4
W20260806 07:54:13.496256 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:13.496177 20502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:13.497550 20492 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
W20260806 07:54:13.497762 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:13.498512 20494 server_base.cc:1061] running on GCE node
I20260806 07:54:13.499279 20494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:13.500650 20494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:13.501875 20494 hybrid_clock.cc:648] HybridClock initialized: now 1786002853501832 us; error 52 us; skew 500 ppm
I20260806 07:54:13.502260 20494 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:13.504444 20494 webserver.cc:533] Webserver started at http://127.20.202.252:34717/ using document root <none> and password file <none>
I20260806 07:54:13.505239 20494 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:13.505401 20494 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:13.505817 20494 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:13.508224 20494 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "334923e9dda94135ba2d75b24229ae9d"
format_stamp: "Formatted at 2026-08-06 07:54:13 on dist-test-slave-g350"
I20260806 07:54:13.509027 20494 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "334923e9dda94135ba2d75b24229ae9d"
format_stamp: "Formatted at 2026-08-06 07:54:13 on dist-test-slave-g350"
I20260806 07:54:13.514298 20494 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 07:54:13.517711 20511 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:13.518999 20494 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:54:13.519192 20494 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "334923e9dda94135ba2d75b24229ae9d"
format_stamp: "Formatted at 2026-08-06 07:54:13 on dist-test-slave-g350"
I20260806 07:54:13.519405 20494 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:13.541117 20494 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:13.542135 20494 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:13.542419 20494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:13.545763 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 3
W20260806 07:54:13.547937 20419 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33575: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33575: connect: Connection refused (error 111)
I20260806 07:54:13.564517 20494 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:33575
I20260806 07:54:13.564534 20562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:33575 every 8 connection(s)
I20260806 07:54:13.566268 20494 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:54:13.570747 20563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:13.574239 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20494
I20260806 07:54:13.574460 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:54:13.578447 20563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:13.594416 20563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:13.600626 20563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:13.611002 20563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: Bootstrap starting.
I20260806 07:54:13.615211 20563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:13.616598 20563 log.cc:826] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:13.619509 20563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: No bootstrap required, opened a new log
I20260806 07:54:13.626399 20563 raft_consensus.cc:359] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.626708 20563 raft_consensus.cc:385] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:13.626811 20563 raft_consensus.cc:740] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 334923e9dda94135ba2d75b24229ae9d, State: Initialized, Role: FOLLOWER
I20260806 07:54:13.627440 20563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.628360 20570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:13.628633 20570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:13.629480 20563 sys_catalog.cc:565] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:13.639416 20581 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:13.639547 20581 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:13.636870 20494 master_runner.cc:187] Error getting master registration for 127.20.202.254:43141: OK, instance_id { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" instance_seqno: 1786002852606453 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:13.643510 20494 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:13.725162 20419 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 4
I20260806 07:54:13.737061 20419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Bootstrap starting.
I20260806 07:54:13.741389 20419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:13.743000 20419 log.cc:826] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:13.746266 20419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: No bootstrap required, opened a new log
I20260806 07:54:13.754029 20419 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.754388 20419 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:13.754531 20419 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8fecf6baec490293b4d8f2e7365894, State: Initialized, Role: FOLLOWER
I20260806 07:54:13.755268 20419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.757608 20584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:13.758042 20584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:13.758991 20419 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:13.769174 20595 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:13.769399 20595 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:13.768750 20353 master_runner.cc:187] Error getting master registration for 127.20.202.253:36167: OK, instance_id { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" instance_seqno: 1786002853088723 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:13.776960 20353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:54:13.794726 20517 tablet.cc:2406] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:13.796926 20492 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } attempt: 5
I20260806 07:54:13.810787 20492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: Bootstrap starting.
I20260806 07:54:13.815210 20492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:13.816666 20492 log.cc:826] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:13.820010 20492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: No bootstrap required, opened a new log
I20260806 07:54:13.829684 20492 raft_consensus.cc:359] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.830051 20492 raft_consensus.cc:385] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:13.830179 20492 raft_consensus.cc:740] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60aa77b18ef04d748526dcf23325819b, State: Initialized, Role: FOLLOWER
I20260806 07:54:13.830969 20492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.832056 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:13.832456 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:13.833405 20492 sys_catalog.cc:565] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:13.843281 20373 tablet.cc:2406] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:13.855283 20423 master_runner.cc:264] No leader master found from master 60aa77b18ef04d748526dcf23325819b
I20260806 07:54:13.855443 20423 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:54:13.858759 20609 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:13.859015 20609 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:13.892730 20584 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:54:13.893028 20584 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.895012 20584 leader_election.cc:290] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60aa77b18ef04d748526dcf23325819b (127.20.202.253:36167), 334923e9dda94135ba2d75b24229ae9d (127.20.202.252:33575)
I20260806 07:54:13.896575 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c8fecf6baec490293b4d8f2e7365894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "334923e9dda94135ba2d75b24229ae9d" is_pre_election: true
I20260806 07:54:13.897156 20538 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c8fecf6baec490293b4d8f2e7365894 in term 0.
I20260806 07:54:13.898478 20369 leader_election.cc:304] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c8fecf6baec490293b4d8f2e7365894, 334923e9dda94135ba2d75b24229ae9d; no voters:
I20260806 07:54:13.898497 20467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c8fecf6baec490293b4d8f2e7365894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60aa77b18ef04d748526dcf23325819b" is_pre_election: true
I20260806 07:54:13.899061 20467 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c8fecf6baec490293b4d8f2e7365894 in term 0.
I20260806 07:54:13.899277 20584 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:54:13.899452 20584 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:54:13.899560 20584 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:13.903606 20584 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.904796 20467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c8fecf6baec490293b4d8f2e7365894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60aa77b18ef04d748526dcf23325819b"
I20260806 07:54:13.905025 20467 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:13.908725 20467 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c8fecf6baec490293b4d8f2e7365894 in term 1.
I20260806 07:54:13.909248 20368 leader_election.cc:304] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c8fecf6baec490293b4d8f2e7365894, 60aa77b18ef04d748526dcf23325819b; no voters:
I20260806 07:54:13.914153 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c8fecf6baec490293b4d8f2e7365894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "334923e9dda94135ba2d75b24229ae9d"
I20260806 07:54:13.914390 20538 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:13.918015 20611 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:54:13.918093 20538 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c8fecf6baec490293b4d8f2e7365894 in term 1.
I20260806 07:54:13.918896 20584 leader_election.cc:290] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [CANDIDATE]: Term 1 election: Requested vote from peers 60aa77b18ef04d748526dcf23325819b (127.20.202.253:36167), 334923e9dda94135ba2d75b24229ae9d (127.20.202.252:33575)
I20260806 07:54:13.919757 20611 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 LEADER]: Becoming Leader. State: Replica: 1c8fecf6baec490293b4d8f2e7365894, State: Running, Role: LEADER
I20260806 07:54:13.920405 20611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } }
I20260806 07:54:13.943109 20584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c8fecf6baec490293b4d8f2e7365894. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:13.943455 20584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:13.955822 20614 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:13.958647 20614 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:13.978309 20538 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Refusing update from remote peer 1c8fecf6baec490293b4d8f2e7365894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:13.979498 20584 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:13.987210 20467 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Refusing update from remote peer 1c8fecf6baec490293b4d8f2e7365894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:13.988148 20584 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:14.025765 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c8fecf6baec490293b4d8f2e7365894. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:14.026077 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:14.034962 20584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:14.035218 20584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:14.035416 20615 mvcc.cc:204] Tried to move back new op lower bound from 7315467689884454912 to 7315467689664684032. Current Snapshot: MvccSnapshot[applied={T|T < 7315467689884454912}]
I20260806 07:54:14.039631 20617 mvcc.cc:204] Tried to move back new op lower bound from 7315467689884454912 to 7315467689664684032. Current Snapshot: MvccSnapshot[applied={T|T < 7315467689884454912}]
I20260806 07:54:14.040956 20614 catalog_manager.cc:1383] Generated new cluster ID: 54258a8d017241439b3c1e7dee246730
I20260806 07:54:14.041062 20614 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:14.055025 20570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c8fecf6baec490293b4d8f2e7365894. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:14.055354 20570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:14.062304 20611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:14.062599 20611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:14.066766 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:14.066999 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:14.069391 20616 mvcc.cc:204] Tried to move back new op lower bound from 7315467689884454912 to 7315467689664684032. Current Snapshot: MvccSnapshot[applied={T|T < 7315467689884454912}]
I20260806 07:54:14.070533 20570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } }
I20260806 07:54:14.073747 20570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:14.102756 20614 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:54:14.104758 20614 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:14.137295 20614 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Generated new TSK 0
I20260806 07:54:14.138433 20614 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:14.155763 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:54:14.580190 20623 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:14.580530 20623 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:14.580658 20623 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:14.589270 20623 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:14.589519 20623 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:14.599798 20623 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:14.601764 20623 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:14.603644 20623 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:14.617102 20629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:14.617422 20628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:14.625208 20623 server_base.cc:1061] running on GCE node
W20260806 07:54:14.625741 20631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:14.626657 20623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:14.627983 20623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:14.629233 20623 hybrid_clock.cc:648] HybridClock initialized: now 1786002854629162 us; error 62 us; skew 500 ppm
I20260806 07:54:14.629683 20623 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:14.632368 20623 webserver.cc:533] Webserver started at http://127.20.202.193:43699/ using document root <none> and password file <none>
I20260806 07:54:14.633574 20623 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:14.633761 20623 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:14.634263 20623 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:14.637686 20623 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:14.638834 20623 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:14.642007 20581 catalog_manager.cc:1295] Loaded cluster ID: 54258a8d017241439b3c1e7dee246730
I20260806 07:54:14.642120 20581 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: loading cluster ID for follower catalog manager: success
I20260806 07:54:14.646570 20581 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: acquiring CA information for follower catalog manager: success
I20260806 07:54:14.650010 20581 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:14.650446 20623 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.003s sys 0.005s
I20260806 07:54:14.656642 20638 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:14.658916 20623 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 07:54:14.659127 20623 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:14.659418 20623 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:14.689000 20623 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:14.690045 20623 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:14.690361 20623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:14.691390 20623 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:14.693212 20623 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:14.693339 20623 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:14.693486 20623 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:14.693615 20623 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:14.730518 20623 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37075
I20260806 07:54:14.730690 20750 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37075 every 8 connection(s)
I20260806 07:54:14.732163 20623 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:14.738055 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20623
I20260806 07:54:14.738236 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:54:14.751369 20752 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33575
I20260806 07:54:14.751385 20751 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43141
I20260806 07:54:14.751416 20753 heartbeater.cc:344] Connected to a master server at 127.20.202.253:36167
I20260806 07:54:14.751798 20751 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:14.751803 20752 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:14.751806 20753 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:14.752952 20752 heartbeater.cc:507] Master 127.20.202.252:33575 requested a full tablet report, sending...
I20260806 07:54:14.752986 20753 heartbeater.cc:507] Master 127.20.202.253:36167 requested a full tablet report, sending...
I20260806 07:54:14.753012 20751 heartbeater.cc:507] Master 127.20.202.254:43141 requested a full tablet report, sending...
I20260806 07:54:14.755266 20528 ts_manager.cc:194] Registered new tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:14.755535 20384 ts_manager.cc:194] Registered new tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:14.755597 20456 ts_manager.cc:194] Registered new tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:14.757332 20384 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:47355
I20260806 07:54:14.763712 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:14.766112 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:35485
--webserver_interface=127.20.202.251
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485 with env {}
I20260806 07:54:14.861166 20609 catalog_manager.cc:1295] Loaded cluster ID: 54258a8d017241439b3c1e7dee246730
I20260806 07:54:14.861264 20609 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: loading cluster ID for follower catalog manager: success
I20260806 07:54:14.864082 20609 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: acquiring CA information for follower catalog manager: success
I20260806 07:54:14.866032 20609 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:54:15.118697 20761 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:15.119062 20761 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:15.119200 20761 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:15.127673 20761 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:15.127786 20761 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:15.127905 20761 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:15.127992 20761 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:15.138288 20761 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:35485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:15.140066 20761 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:15.141880 20761 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:15.150048 20766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:15.150074 20767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:15.150826 20769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:15.151603 20761 server_base.cc:1061] running on GCE node
I20260806 07:54:15.152424 20761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:15.154104 20761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:15.155373 20761 hybrid_clock.cc:648] HybridClock initialized: now 1786002855155323 us; error 40 us; skew 500 ppm
I20260806 07:54:15.155792 20761 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:15.158320 20761 webserver.cc:533] Webserver started at http://127.20.202.251:35541/ using document root <none> and password file <none>
I20260806 07:54:15.159214 20761 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:15.159356 20761 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:15.159766 20761 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:15.162500 20761 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "edf871fa899049f1a3faad54243443c0"
format_stamp: "Formatted at 2026-08-06 07:54:15 on dist-test-slave-g350"
I20260806 07:54:15.163411 20761 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "edf871fa899049f1a3faad54243443c0"
format_stamp: "Formatted at 2026-08-06 07:54:15 on dist-test-slave-g350"
I20260806 07:54:15.169047 20761 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:54:15.172585 20775 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:15.174026 20761 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:15.174254 20761 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "edf871fa899049f1a3faad54243443c0"
format_stamp: "Formatted at 2026-08-06 07:54:15 on dist-test-slave-g350"
I20260806 07:54:15.174479 20761 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:15.208047 20761 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:15.209046 20761 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:15.209450 20761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:15.231645 20761 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:35485
I20260806 07:54:15.231670 20826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:35485 every 8 connection(s)
I20260806 07:54:15.233430 20761 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:15.236430 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20761
I20260806 07:54:15.238477 20827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:15.247112 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:15.257665 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:15.264635 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:15.271219 20827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:15.281407 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Bootstrap starting.
I20260806 07:54:15.284633 20827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:15.285810 20827 log.cc:826] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:15.292076 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: No bootstrap required, opened a new log
I20260806 07:54:15.297314 20827 raft_consensus.cc:359] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } }
I20260806 07:54:15.297808 20827 raft_consensus.cc:385] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:15.298013 20827 raft_consensus.cc:740] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf871fa899049f1a3faad54243443c0, State: Initialized, Role: FOLLOWER
I20260806 07:54:15.298688 20827 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } }
I20260806 07:54:15.299683 20836 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } } }
I20260806 07:54:15.300148 20836 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:15.301060 20827 sys_catalog.cc:565] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:15.311723 20847 catalog_manager.cc:1594] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:15.311926 20847 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:15.319850 20761 master_runner.cc:429] Detected that this master edf871fa899049f1a3faad54243443c0 is joining an existing cluster
I20260806 07:54:15.319999 20761 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:54:15.320894 20761 raft_consensus.cc:2243] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:54:15.321280 20761 raft_consensus.cc:2272] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:54:15.321489 20761 tablet_replica.cc:333] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: stopping tablet replica
I20260806 07:54:15.323536 20761 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:35485
I20260806 07:54:15.361195 20384 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:35485: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "edf871fa899049f1a3faad54243443c0"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 35485
}
attrs {
promote: true
}
}
dest_uuid: "1c8fecf6baec490293b4d8f2e7365894"
cas_config_opid_index: -1
I20260806 07:54:15.363023 20384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:15.366385 20466 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Refusing update from remote peer 1c8fecf6baec490293b4d8f2e7365894: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:15.367121 20636 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260806 07:54:15.367486 20369 consensus_peers.cc:597] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 -> Peer edf871fa899049f1a3faad54243443c0 (127.20.202.251:35485): Couldn't send request to peer edf871fa899049f1a3faad54243443c0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:54:15.370288 20636 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER edf871fa899049f1a3faad54243443c0 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.370760 20466 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER edf871fa899049f1a3faad54243443c0 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.376147 20537 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Refusing update from remote peer 1c8fecf6baec490293b4d8f2e7365894: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:15.378114 20861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.378511 20861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:15.382071 20863 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:15.382766 20636 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:35485
I20260806 07:54:15.383348 20859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.383591 20859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:15.383764 20761 master.cc:562] Master@127.20.202.251:35485 shutting down...
I20260806 07:54:15.388252 20636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.388670 20636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:15.390024 20537 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER edf871fa899049f1a3faad54243443c0 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.393698 20866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.394074 20866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:54:15.420702 20350 connection.cc:573] client connection to 127.20.202.251:35485 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:15.433913 20761 master.cc:584] Master@127.20.202.251:35485 shutdown complete.
I20260806 07:54:15.434048 20761 master_runner.cc:306] Clearing existing system tablet
I20260806 07:54:15.434468 20761 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:15.434593 20761 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:15.437521 20761 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:54:15.439344 20868 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:15.439723 20761 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:15.440006 20761 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 07:54:15.440124 20761 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "edf871fa899049f1a3faad54243443c0"
format_stamp: "Formatted at 2026-08-06 07:54:15 on dist-test-slave-g350"
I20260806 07:54:15.444728 20761 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:54:15.452049 20761 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:54:15.452225 20761 log.cc:1199] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:54:15.452750 20761 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Deleting consensus metadata
I20260806 07:54:15.453012 20761 master_runner.cc:316] Copying system tablet from 127.20.202.254:43141
I20260806 07:54:15.455884 20761 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.254:43141
I20260806 07:54:15.463505 20404 tablet_copy_service.cc:140] P 1c8fecf6baec490293b4d8f2e7365894: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer edf871fa899049f1a3faad54243443c0 ({username='slave'} at 127.0.0.1:52598)
I20260806 07:54:15.463778 20404 tablet_copy_service.cc:161] P 1c8fecf6baec490293b4d8f2e7365894: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer edf871fa899049f1a3faad54243443c0 at {username='slave'} at 127.0.0.1:52598: session id = edf871fa899049f1a3faad54243443c0-00000000000000000000000000000000
I20260806 07:54:15.466849 20404 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:54:15.469013 20761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:54:15.476372 20761 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: tablet copy: Starting download of 0 data blocks...
I20260806 07:54:15.476675 20761 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: tablet copy: Starting download of 1 WAL segments...
I20260806 07:54:15.479368 20761 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:54:15.484030 20404 tablet_copy_service.cc:342] P 1c8fecf6baec490293b4d8f2e7365894: Request end of tablet copy session edf871fa899049f1a3faad54243443c0-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:52598
I20260806 07:54:15.484262 20404 tablet_copy_service.cc:434] P 1c8fecf6baec490293b4d8f2e7365894: ending tablet copy session edf871fa899049f1a3faad54243443c0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer edf871fa899049f1a3faad54243443c0
W20260806 07:54:15.491261 20761 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:54:15.495375 20761 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:15.498050 20875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:15.498211 20876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:15.498879 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:15.499768 20761 server_base.cc:1061] running on GCE node
I20260806 07:54:15.500139 20761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:15.500790 20761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:15.502018 20761 hybrid_clock.cc:648] HybridClock initialized: now 1786002855501902 us; error 78 us; skew 500 ppm
I20260806 07:54:15.502229 20761 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:15.503444 20761 webserver.cc:533] Webserver started at http://127.20.202.251:45335/ using document root <none> and password file <none>
I20260806 07:54:15.503783 20761 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:15.503886 20761 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:15.506100 20761 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 07:54:15.507709 20884 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:15.508107 20761 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 07:54:15.508276 20761 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "edf871fa899049f1a3faad54243443c0"
format_stamp: "Formatted at 2026-08-06 07:54:15 on dist-test-slave-g350"
I20260806 07:54:15.508437 20761 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:15.525230 20761 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:15.526003 20761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:15.544570 20761 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:35485
I20260806 07:54:15.544612 20936 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:35485 every 8 connection(s)
I20260806 07:54:15.545521 20761 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:15.550505 20937 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:15.554116 20937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Bootstrap starting.
I20260806 07:54:15.572158 20937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:15.572732 20937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: Bootstrap complete.
I20260806 07:54:15.573904 20937 raft_consensus.cc:359] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:15.574163 20937 raft_consensus.cc:740] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: edf871fa899049f1a3faad54243443c0, State: Initialized, Role: LEARNER
I20260806 07:54:15.574431 20937 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:15.578161 20937 sys_catalog.cc:565] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:15.577958 20940 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.578321 20940 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:15.588749 20952 catalog_manager.cc:1295] Loaded cluster ID: 54258a8d017241439b3c1e7dee246730
I20260806 07:54:15.588873 20952 catalog_manager.cc:1588] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: loading cluster ID for follower catalog manager: success
I20260806 07:54:15.588922 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20623
I20260806 07:54:15.593856 20952 catalog_manager.cc:1610] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: acquiring CA information for follower catalog manager: success
I20260806 07:54:15.596256 20952 catalog_manager.cc:1638] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:15.602015 20746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:15.715231 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20623
I20260806 07:54:15.734206 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37075
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--webserver_port=43699
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485 with env {}
I20260806 07:54:15.805080 20912 raft_consensus.cc:1217] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:54:15.805869 20940 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c8fecf6baec490293b4d8f2e7365894. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.806121 20940 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:15.894734 20863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:15.895095 20863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
W20260806 07:54:16.099799 20955 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:16.100201 20955 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:16.100319 20955 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:16.113840 20955 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:16.114001 20955 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:16.124369 20955 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37075
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43699
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:16.126039 20955 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:16.127874 20955 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:16.138567 20964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:16.138618 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:16.138721 20966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:16.139135 20955 server_base.cc:1061] running on GCE node
I20260806 07:54:16.139992 20955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:16.140885 20955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:16.142168 20955 hybrid_clock.cc:648] HybridClock initialized: now 1786002856142102 us; error 64 us; skew 500 ppm
I20260806 07:54:16.142504 20955 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:16.144621 20955 webserver.cc:533] Webserver started at http://127.20.202.193:43699/ using document root <none> and password file <none>
I20260806 07:54:16.145385 20955 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:16.145550 20955 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:16.151228 20955 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 07:54:16.153915 20972 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:16.155342 20955 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:54:16.155526 20955 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:16.156267 20955 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:16.167846 20955 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:16.168664 20955 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:16.168959 20955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:16.169901 20955 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:16.171526 20955 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:16.171639 20955 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:16.171772 20955 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:16.171839 20955 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:16.205350 20955 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37075
I20260806 07:54:16.205394 21086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37075 every 8 connection(s)
I20260806 07:54:16.206911 20955 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:16.215207 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 20955
I20260806 07:54:16.227495 21088 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43141
I20260806 07:54:16.227524 21087 heartbeater.cc:344] Connected to a master server at 127.20.202.251:35485
I20260806 07:54:16.228133 21087 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:16.228135 21088 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:16.229440 21088 heartbeater.cc:507] Master 127.20.202.254:43141 requested a full tablet report, sending...
I20260806 07:54:16.229467 21087 heartbeater.cc:507] Master 127.20.202.251:35485 requested a full tablet report, sending...
I20260806 07:54:16.231650 20384 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:16.232455 20902 ts_manager.cc:194] Registered new tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:16.232743 21090 heartbeater.cc:344] Connected to a master server at 127.20.202.253:36167
I20260806 07:54:16.232795 20384 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:35649
I20260806 07:54:16.232887 21090 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:16.233471 21090 heartbeater.cc:507] Master 127.20.202.253:36167 requested a full tablet report, sending...
I20260806 07:54:16.235028 21089 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33575
I20260806 07:54:16.235201 21089 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:16.235517 21089 heartbeater.cc:507] Master 127.20.202.252:33575 requested a full tablet report, sending...
I20260806 07:54:16.236368 20456 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:16.236848 20528 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:16.243120 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:16.250597 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:16.250783 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20353
I20260806 07:54:16.251641 20414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:16.367035 20956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c8fecf6baec490293b4d8f2e7365894" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:16.367312 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20353
I20260806 07:54:16.367357 20956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: LEADER
W20260806 07:54:16.391345 20887 connection.cc:573] server connection from 127.0.0.1:40834 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:16.392302 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43141
--webserver_interface=127.20.202.254
--webserver_port=41095
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485 with env {}
W20260806 07:54:16.757469 21100 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:16.757930 21100 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:16.758121 21100 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:16.767105 21100 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:16.767207 21100 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:16.767293 21100 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:16.767373 21100 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:16.778043 21100 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=41095
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:16.779874 21100 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:16.782548 21100 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:16.792125 21106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:16.792299 21105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:16.792153 21108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:16.793552 21100 server_base.cc:1061] running on GCE node
I20260806 07:54:16.794442 21100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:16.795984 21100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:16.797255 21100 hybrid_clock.cc:648] HybridClock initialized: now 1786002856797212 us; error 33 us; skew 500 ppm
I20260806 07:54:16.797720 21100 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:16.800153 21100 webserver.cc:533] Webserver started at http://127.20.202.254:41095/ using document root <none> and password file <none>
I20260806 07:54:16.800951 21100 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:16.801075 21100 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:16.806897 21100 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:54:16.810053 21114 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:16.811558 21100 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:54:16.811718 21100 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "1c8fecf6baec490293b4d8f2e7365894"
format_stamp: "Formatted at 2026-08-06 07:54:12 on dist-test-slave-g350"
I20260806 07:54:16.812623 21100 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:16.826815 21100 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:16.827776 21100 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:16.828088 21100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:16.850939 21100 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43141
I20260806 07:54:16.850988 21166 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43141 every 8 connection(s)
I20260806 07:54:16.852720 21100 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:16.853793 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21100
I20260806 07:54:16.854135 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20955
I20260806 07:54:16.862679 21167 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:16.866896 21167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Bootstrap starting.
I20260806 07:54:16.868803 21082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:16.887283 21167 log.cc:826] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:16.897228 21167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:16.897946 21167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: Bootstrap complete.
I20260806 07:54:16.905715 21167 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:16.906508 21167 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8fecf6baec490293b4d8f2e7365894, State: Initialized, Role: FOLLOWER
I20260806 07:54:16.907390 21167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:16.908313 21173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:16.908653 21173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:16.909631 21167 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:16.926462 21184 catalog_manager.cc:1295] Loaded cluster ID: 54258a8d017241439b3c1e7dee246730
I20260806 07:54:16.928960 21184 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: loading cluster ID for follower catalog manager: success
I20260806 07:54:16.935276 21184 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: acquiring CA information for follower catalog manager: success
I20260806 07:54:16.938648 21184 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:16.972234 21100 master_runner.cc:264] No leader master found from master 1c8fecf6baec490293b4d8f2e7365894
I20260806 07:54:16.972429 21100 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:16.983546 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20955
I20260806 07:54:17.003249 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37075
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--webserver_port=43699
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485 with env {}
I20260806 07:54:17.375283 21192 raft_consensus.cc:493] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1c8fecf6baec490293b4d8f2e7365894)
I20260806 07:54:17.375646 21192 raft_consensus.cc:515] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:17.377117 21192 leader_election.cc:290] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c8fecf6baec490293b4d8f2e7365894 (127.20.202.254:43141), 334923e9dda94135ba2d75b24229ae9d (127.20.202.252:33575)
I20260806 07:54:17.378069 20537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60aa77b18ef04d748526dcf23325819b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "334923e9dda94135ba2d75b24229ae9d" is_pre_election: true
I20260806 07:54:17.385313 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60aa77b18ef04d748526dcf23325819b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1c8fecf6baec490293b4d8f2e7365894" is_pre_election: true
I20260806 07:54:17.386013 21142 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60aa77b18ef04d748526dcf23325819b in term 1.
I20260806 07:54:17.386659 20440 leader_election.cc:304] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c8fecf6baec490293b4d8f2e7365894, 60aa77b18ef04d748526dcf23325819b; no voters: 334923e9dda94135ba2d75b24229ae9d
I20260806 07:54:17.387100 21192 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:54:17.387236 21192 raft_consensus.cc:493] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1c8fecf6baec490293b4d8f2e7365894)
I20260806 07:54:17.387341 21192 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:54:17.390645 21192 raft_consensus.cc:515] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:17.391425 21192 leader_election.cc:290] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [CANDIDATE]: Term 2 election: Requested vote from peers 1c8fecf6baec490293b4d8f2e7365894 (127.20.202.254:43141), 334923e9dda94135ba2d75b24229ae9d (127.20.202.252:33575)
I20260806 07:54:17.392030 20537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60aa77b18ef04d748526dcf23325819b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "334923e9dda94135ba2d75b24229ae9d"
I20260806 07:54:17.392050 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60aa77b18ef04d748526dcf23325819b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1c8fecf6baec490293b4d8f2e7365894"
I20260806 07:54:17.392244 21142 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:54:17.396623 21197 raft_consensus.cc:493] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1c8fecf6baec490293b4d8f2e7365894)
I20260806 07:54:17.396852 21197 raft_consensus.cc:515] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:17.397836 21142 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60aa77b18ef04d748526dcf23325819b in term 2.
W20260806 07:54:17.398031 21191 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
I20260806 07:54:17.398344 21197 leader_election.cc:290] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c8fecf6baec490293b4d8f2e7365894 (127.20.202.254:43141), 60aa77b18ef04d748526dcf23325819b (127.20.202.253:36167)
W20260806 07:54:17.398485 21191 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260806 07:54:17.398442 20440 leader_election.cc:304] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c8fecf6baec490293b4d8f2e7365894, 60aa77b18ef04d748526dcf23325819b; no voters: 334923e9dda94135ba2d75b24229ae9d
W20260806 07:54:17.398675 21191 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:54:17.398967 21192 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:54:17.399602 20466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "334923e9dda94135ba2d75b24229ae9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "60aa77b18ef04d748526dcf23325819b" is_pre_election: true
I20260806 07:54:17.400104 21192 raft_consensus.cc:697] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 2 LEADER]: Becoming Leader. State: Replica: 60aa77b18ef04d748526dcf23325819b, State: Running, Role: LEADER
I20260806 07:54:17.400712 21192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } }
I20260806 07:54:17.403142 21199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60aa77b18ef04d748526dcf23325819b. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.403466 21199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:17.405124 21201 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:17.407300 21201 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:17.407433 21142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "334923e9dda94135ba2d75b24229ae9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1c8fecf6baec490293b4d8f2e7365894" is_pre_election: true
I20260806 07:54:17.407819 21142 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 334923e9dda94135ba2d75b24229ae9d in current term 2: Already voted for candidate 60aa77b18ef04d748526dcf23325819b in this term.
I20260806 07:54:17.408253 21201 catalog_manager.cc:1295] Loaded cluster ID: 54258a8d017241439b3c1e7dee246730
I20260806 07:54:17.408349 21201 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:17.408661 20512 leader_election.cc:304] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 334923e9dda94135ba2d75b24229ae9d; no voters: 1c8fecf6baec490293b4d8f2e7365894, 60aa77b18ef04d748526dcf23325819b
I20260806 07:54:17.409385 21197 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:54:17.409888 21201 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:17.410938 21201 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: Loaded TSK: 0
W20260806 07:54:17.411209 21191 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:17.411420 21191 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:17.412124 21201 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:17.412565 21197 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:54:17.422214 21191 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37075
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43699
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:17.423920 21191 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:17.425917 21191 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:17.435941 21203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:17.435917 21205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:17.437427 21207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:17.437922 21191 server_base.cc:1061] running on GCE node
I20260806 07:54:17.438539 21191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:17.439673 21191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:17.440912 21191 hybrid_clock.cc:648] HybridClock initialized: now 1786002857440845 us; error 59 us; skew 500 ppm
I20260806 07:54:17.441254 21191 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:17.443543 21191 webserver.cc:533] Webserver started at http://127.20.202.193:43699/ using document root <none> and password file <none>
I20260806 07:54:17.444357 21191 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:17.444496 21191 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:17.450275 21191 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:54:17.453058 21213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:17.454859 21191 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:17.455060 21191 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:17.455858 21191 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:17.494316 21191 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:17.495265 21191 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:17.495575 21191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:17.496619 21191 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:17.498383 21191 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:17.498488 21191 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:17.498652 21191 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:17.498726 21191 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:17.535238 21191 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37075
I20260806 07:54:17.535336 21331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37075 every 8 connection(s)
I20260806 07:54:17.536890 21191 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:17.544657 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21191
I20260806 07:54:17.544982 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21191
I20260806 07:54:17.561414 21333 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43141
I20260806 07:54:17.561908 21333 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:17.563601 21333 heartbeater.cc:507] Master 127.20.202.254:43141 requested a full tablet report, sending...
I20260806 07:54:17.567615 21132 ts_manager.cc:194] Registered new tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:17.570760 21332 heartbeater.cc:344] Connected to a master server at 127.20.202.251:35485
I20260806 07:54:17.571033 21332 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:17.571810 21332 heartbeater.cc:507] Master 127.20.202.251:35485 requested a full tablet report, sending...
I20260806 07:54:17.573315 20902 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:17.578299 21327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:17.666810 21334 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33575
I20260806 07:54:17.667225 21334 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:17.667944 21334 heartbeater.cc:507] Master 127.20.202.252:33575 requested a full tablet report, sending...
I20260806 07:54:17.668478 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21191
I20260806 07:54:17.670326 20528 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
W20260806 07:54:17.687284 20512 connection.cc:573] server connection from 127.20.202.193:59563 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:17.688210 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37075
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--webserver_port=43699
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--fault_before_start=1.0 with env {}
I20260806 07:54:17.784788 20537 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 2 FOLLOWER]: Refusing update from remote peer 60aa77b18ef04d748526dcf23325819b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:54:17.785657 21199 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:17.787827 21197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60aa77b18ef04d748526dcf23325819b. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.788326 21197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:17.791443 21192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.791811 21192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:17.792214 21142 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 2 FOLLOWER]: Refusing update from remote peer 60aa77b18ef04d748526dcf23325819b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:54:17.792801 21197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.793025 21197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:17.793710 21349 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:17.797961 20912 raft_consensus.cc:3060] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 1 LEARNER]: Advancing to term 2
I20260806 07:54:17.798501 21354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.798897 21354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:17.799347 21351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60aa77b18ef04d748526dcf23325819b. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.799592 21351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:17.801559 20912 raft_consensus.cc:1275] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 2 LEARNER]: Refusing update from remote peer 60aa77b18ef04d748526dcf23325819b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:54:17.802333 21349 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:17.806067 21358 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.806519 21358 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:17.806519 21355 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60aa77b18ef04d748526dcf23325819b. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: true } } }
I20260806 07:54:17.806850 21349 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b: attempting to promote NON_VOTER edf871fa899049f1a3faad54243443c0 to VOTER
I20260806 07:54:17.806874 21355 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:17.808285 21349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } }
I20260806 07:54:17.811017 20912 raft_consensus.cc:1275] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 2 LEARNER]: Refusing update from remote peer 60aa77b18ef04d748526dcf23325819b: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:54:17.811204 21142 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 2 FOLLOWER]: Refusing update from remote peer 60aa77b18ef04d748526dcf23325819b: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:54:17.811219 20537 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 2 FOLLOWER]: Refusing update from remote peer 60aa77b18ef04d748526dcf23325819b: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:54:17.811877 21192 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:54:17.812237 21199 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:54:17.812453 21359 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:54:17.815676 21192 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, edf871fa899049f1a3faad54243443c0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.816614 20537 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, edf871fa899049f1a3faad54243443c0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.816457 20912 raft_consensus.cc:2955] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, edf871fa899049f1a3faad54243443c0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.820293 21199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.820513 21199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:17.820596 21192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.820837 21192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:17.820770 21199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.820933 21199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:17.821080 21192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.821136 21355 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.821282 21192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa77b18ef04d748526dcf23325819b [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:17.821333 21355 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf871fa899049f1a3faad54243443c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:17.822273 21142 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, edf871fa899049f1a3faad54243443c0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.826104 21197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.826401 21197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334923e9dda94135ba2d75b24229ae9d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:17.827050 21351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "60aa77b18ef04d748526dcf23325819b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1c8fecf6baec490293b4d8f2e7365894" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43141 } } peers { permanent_uuid: "60aa77b18ef04d748526dcf23325819b" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 36167 } } peers { permanent_uuid: "334923e9dda94135ba2d75b24229ae9d" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33575 } } peers { permanent_uuid: "edf871fa899049f1a3faad54243443c0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35485 } attrs { promote: false } } }
I20260806 07:54:17.827280 21351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fecf6baec490293b4d8f2e7365894 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:54:18.063881 21346 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:18.064203 21346 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:18.064285 21346 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:54:18.064342 21346 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:18.072975 21346 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:18.073184 21346 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:18.083848 21346 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37075
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43699
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:18.085583 21346 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:18.087524 21346 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:18.097532 21365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:18.097791 21364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:18.099130 21346 server_base.cc:1061] running on GCE node
W20260806 07:54:18.099364 21367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:18.100071 21346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:18.101114 21346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:18.102362 21346 hybrid_clock.cc:648] HybridClock initialized: now 1786002858102272 us; error 57 us; skew 500 ppm
I20260806 07:54:18.102722 21346 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:18.105232 21346 webserver.cc:533] Webserver started at http://127.20.202.193:43699/ using document root <none> and password file <none>
I20260806 07:54:18.106173 21346 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:18.106304 21346 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:18.113000 21346 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:54:18.116025 21373 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:18.117446 21346 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:54:18.117659 21346 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:18.118474 21346 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:18.144814 21346 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:18.145840 21346 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:18.146148 21346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:18.147200 21346 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:18.149060 21346 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:18.149147 21346 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:18.149272 21346 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:18.149358 21346 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:54:18.149821 21346 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:54:18.149945 21346 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:54:18.150024 21346 fault_injection.cc:59] injecting fault for kudu (pid 21346): FLAGS_fault_before_start (process will exit)
I20260806 07:54:18.171736 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37075
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575
--webserver_port=43699
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485 with env {}
W20260806 07:54:18.529978 21381 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:18.530354 21381 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:18.530494 21381 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:18.539505 21381 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:18.539726 21381 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:18.554075 21381 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37075
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43699
--tserver_master_addrs=127.20.202.254:43141,127.20.202.253:36167,127.20.202.252:33575,127.20.202.251:35485
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:18.556217 21381 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:18.558542 21381 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:18.568825 21387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:18.568908 21386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:18.570955 21389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:18.571595 21381 server_base.cc:1061] running on GCE node
I20260806 07:54:18.572271 21381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:18.573407 21381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:18.574658 21381 hybrid_clock.cc:648] HybridClock initialized: now 1786002858574590 us; error 58 us; skew 500 ppm
I20260806 07:54:18.575019 21381 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:18.577721 21381 webserver.cc:533] Webserver started at http://127.20.202.193:43699/ using document root <none> and password file <none>
I20260806 07:54:18.578687 21381 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:18.578804 21381 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:18.585525 21381 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 07:54:18.588388 21395 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:18.589989 21381 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:18.590142 21381 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "dacc8ece67424301975a6ae79a3e2e10"
format_stamp: "Formatted at 2026-08-06 07:54:14 on dist-test-slave-g350"
I20260806 07:54:18.590840 21381 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:18.622435 21381 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:18.623324 21381 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:18.623584 21381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:18.624610 21381 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:18.626354 21381 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:18.626427 21381 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:18.626523 21381 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:18.626569 21381 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:18.665863 21381 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37075
I20260806 07:54:18.665884 21507 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37075 every 8 connection(s)
I20260806 07:54:18.667517 21381 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:18.674255 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21381
I20260806 07:54:18.674546 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21381
I20260806 07:54:18.702955 21509 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43141
I20260806 07:54:18.703120 21508 heartbeater.cc:344] Connected to a master server at 127.20.202.251:35485
I20260806 07:54:18.703457 21508 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:18.704768 21508 heartbeater.cc:507] Master 127.20.202.251:35485 requested a full tablet report, sending...
I20260806 07:54:18.703454 21509 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:18.706532 21509 heartbeater.cc:507] Master 127.20.202.254:43141 requested a full tablet report, sending...
I20260806 07:54:18.707371 21503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:18.708138 21132 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:18.708623 20902 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:18.796120 21517 heartbeater.cc:344] Connected to a master server at 127.20.202.253:36167
I20260806 07:54:18.796408 21517 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:18.797307 21517 heartbeater.cc:507] Master 127.20.202.253:36167 requested a full tablet report, sending...
I20260806 07:54:18.797958 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21381
I20260806 07:54:18.799368 20456 ts_manager.cc:194] Re-registered known tserver with Master: dacc8ece67424301975a6ae79a3e2e10 (127.20.202.193:37075)
I20260806 07:54:18.801127 20456 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:43325
W20260806 07:54:18.817955 20441 connection.cc:573] server connection from 127.20.202.193:43325 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:18.818054 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21100
W20260806 07:54:18.818943 21521 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:54:18.688756 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:33441 (local address 127.20.202.252:33575)
0806 07:54:18.696444 (+ 7688us) server_negotiation.cc:207] Beginning negotiation
0806 07:54:18.696450 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:54:18.699639 (+ 3189us) server_negotiation.cc:408] Connection header received
0806 07:54:18.699716 (+ 77us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:54:18.699722 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:54:18.699810 (+ 88us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:54:18.699920 (+ 110us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:54:18.702816 (+ 2896us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:54:18.704833 (+ 2017us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:54:18.797630 (+ 92797us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:54:18.798159 (+ 529us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:54:18.817930 (+ 19771us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:33441: BlockingRecv error: recv got EOF from 127.20.202.193:33441 (error 108)
Metrics: {"server-negotiator.queue_time_us":7572,"thread_start_us":7461,"threads_started":1}
I20260806 07:54:18.824771 21162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:18.935307 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21100
W20260806 07:54:18.959219 20440 connection.cc:573] client connection to 127.20.202.254:43141 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:18.959481 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20423
I20260806 07:54:18.959568 20423 master_runner.cc:217] Error getting master consensus for 127.20.202.254:43141: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:18.959854 20423 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:18.960695 20487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:19.074237 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20423
I20260806 07:54:19.098327 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20494
I20260806 07:54:19.099612 20558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:19.211200 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20494
I20260806 07:54:19.233758 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 20761
I20260806 07:54:19.234990 20932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:19.349080 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 20761
2026-08-06T07:54:19Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/1 (7299 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/2
2026-08-06T07:54:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:19Z Disabled control of system clock
2026-08-06T07:54:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:19Z Disabled control of system clock
2026-08-06T07:54:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:19Z Disabled control of system clock
2026-08-06T07:54:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:19Z Disabled control of system clock
2026-08-06T07:54:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:19Z Disabled control of system clock
I20260806 07:54:19.626734 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43533
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113 with env {}
W20260806 07:54:19.992622 21538 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:19.992976 21538 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:19.993129 21538 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:20.001881 21538 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:20.001991 21538 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:20.002084 21538 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:20.002182 21538 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:20.012903 21538 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:20.014708 21538 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:20.016673 21538 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:20.025350 21544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:20.025928 21543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:20.026886 21546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:20.027426 21538 server_base.cc:1061] running on GCE node
I20260806 07:54:20.028247 21538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:20.030052 21538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:20.031292 21538 hybrid_clock.cc:648] HybridClock initialized: now 1786002860031164 us; error 93 us; skew 500 ppm
I20260806 07:54:20.031687 21538 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:20.034051 21538 webserver.cc:533] Webserver started at http://127.20.202.254:43069/ using document root <none> and password file <none>
I20260806 07:54:20.034942 21538 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:20.035115 21538 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:20.035475 21538 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:20.038112 21538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "ec199fe4744d4e43818dcd65b7601095"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:20.038898 21538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "ec199fe4744d4e43818dcd65b7601095"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:20.044803 21538 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:54:20.048632 21552 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:20.050276 21538 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:20.050477 21538 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "ec199fe4744d4e43818dcd65b7601095"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:20.050711 21538 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:20.093748 21538 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:20.094802 21538 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:20.095155 21538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:20.119028 21538 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43533
I20260806 07:54:20.119064 21603 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43533 every 8 connection(s)
I20260806 07:54:20.120852 21538 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:20.125415 21604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:20.126700 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21538
I20260806 07:54:20.126946 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:54:20.128052 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45099
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113 with env {}
I20260806 07:54:20.131836 21604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:20.150087 21604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:20.151310 21555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45099: connect: Connection refused (error 111)
W20260806 07:54:20.153009 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45099: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45099: connect: Connection refused (error 111)
I20260806 07:54:20.202433 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } attempt: 1
W20260806 07:54:20.204648 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45099: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45099: connect: Connection refused (error 111)
I20260806 07:54:20.272982 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } attempt: 2
W20260806 07:54:20.275326 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45099: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45099: connect: Connection refused (error 111)
I20260806 07:54:20.366684 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } attempt: 3
W20260806 07:54:20.369127 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45099: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45099: connect: Connection refused (error 111)
W20260806 07:54:20.491086 21608 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:20.491423 21608 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:20.491497 21608 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:20.500051 21608 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:20.500150 21608 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:20.500205 21608 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:20.500247 21608 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:20.510445 21608 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:20.512183 21608 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:20.512440 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } attempt: 4
I20260806 07:54:20.514755 21608 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:20.514887 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45099: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45099: connect: Connection refused (error 111)
W20260806 07:54:20.523460 21614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:20.523491 21615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:20.523535 21617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:20.524770 21608 server_base.cc:1061] running on GCE node
I20260806 07:54:20.525564 21608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:20.527269 21608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:20.528501 21608 hybrid_clock.cc:648] HybridClock initialized: now 1786002860528341 us; error 127 us; skew 500 ppm
I20260806 07:54:20.528863 21608 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:20.531126 21608 webserver.cc:533] Webserver started at http://127.20.202.253:41959/ using document root <none> and password file <none>
I20260806 07:54:20.531976 21608 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:20.532112 21608 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:20.532454 21608 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:20.535243 21608 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "1f48d59f61bb48fe8e841a434e516672"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:20.536321 21608 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "1f48d59f61bb48fe8e841a434e516672"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:20.542234 21608 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:54:20.545799 21623 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:20.547446 21608 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:54:20.547664 21608 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "1f48d59f61bb48fe8e841a434e516672"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:20.547962 21608 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:20.571683 21608 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:20.572817 21608 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:20.573134 21608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:20.596560 21608 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:45099
I20260806 07:54:20.596586 21675 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:45099 every 8 connection(s)
I20260806 07:54:20.598315 21608 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:54:20.603123 21676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:20.603575 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21608
I20260806 07:54:20.603790 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:54:20.605425 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:45113
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113 with env {}
I20260806 07:54:20.613335 21676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:20.629896 21676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:20.640672 21676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:20.642031 21626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
W20260806 07:54:20.643957 21676 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:20.693353 21676 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 1
W20260806 07:54:20.696228 21676 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:20.764505 21676 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 2
W20260806 07:54:20.767036 21676 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:20.814198 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } attempt: 5
I20260806 07:54:20.821996 21604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:20.823943 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:20.858323 21676 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 3
W20260806 07:54:20.860891 21676 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:20.875196 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 1
W20260806 07:54:20.877794 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:20.946036 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 2
W20260806 07:54:20.948321 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
W20260806 07:54:20.977977 21679 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:20.978358 21679 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:20.978503 21679 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:20.987074 21679 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:20.987223 21679 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:20.987337 21679 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:20.987419 21679 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:20.998314 21679 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:45113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:21.000164 21679 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:21.002198 21679 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:54:21.004117 21676 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 4
W20260806 07:54:21.006374 21676 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
W20260806 07:54:21.011638 21687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:21.011880 21689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:21.011636 21686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:21.012694 21679 server_base.cc:1061] running on GCE node
I20260806 07:54:21.013442 21679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:21.015260 21679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:21.016489 21679 hybrid_clock.cc:648] HybridClock initialized: now 1786002861016348 us; error 106 us; skew 500 ppm
I20260806 07:54:21.016914 21679 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:21.019309 21679 webserver.cc:533] Webserver started at http://127.20.202.252:35907/ using document root <none> and password file <none>
I20260806 07:54:21.020130 21679 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:21.020256 21679 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:21.020649 21679 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:21.023329 21679 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "cebfacc2d8b64fd299a42e9ed8865b80"
format_stamp: "Formatted at 2026-08-06 07:54:21 on dist-test-slave-g350"
I20260806 07:54:21.024149 21679 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "cebfacc2d8b64fd299a42e9ed8865b80"
format_stamp: "Formatted at 2026-08-06 07:54:21 on dist-test-slave-g350"
I20260806 07:54:21.029743 21679 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:54:21.033370 21695 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:21.034945 21679 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:21.035130 21679 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "cebfacc2d8b64fd299a42e9ed8865b80"
format_stamp: "Formatted at 2026-08-06 07:54:21 on dist-test-slave-g350"
I20260806 07:54:21.035364 21679 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:21.053158 21679 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:21.054291 21679 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:21.054515 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 3
I20260806 07:54:21.054612 21679 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260806 07:54:21.056852 21604 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:45113: Network error: Client connection negotiation failed: client connection to 127.20.202.252:45113: connect: Connection refused (error 111)
I20260806 07:54:21.076169 21679 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:45113
I20260806 07:54:21.076201 21746 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:45113 every 8 connection(s)
I20260806 07:54:21.077970 21679 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:54:21.082715 21747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:21.083348 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21679
I20260806 07:54:21.083588 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:54:21.091112 21747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:21.104286 21747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:21.111969 21747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:21.123268 21747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: Bootstrap starting.
I20260806 07:54:21.127012 21747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:21.128139 21747 log.cc:826] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:21.130995 21747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: No bootstrap required, opened a new log
I20260806 07:54:21.136615 21747 raft_consensus.cc:359] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.136930 21747 raft_consensus.cc:385] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:21.137066 21747 raft_consensus.cc:740] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebfacc2d8b64fd299a42e9ed8865b80, State: Initialized, Role: FOLLOWER
I20260806 07:54:21.137702 21747 consensus_queue.cc:260] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.138584 21754 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.138831 21754 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.139772 21747 sys_catalog.cc:565] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:21.151412 21765 catalog_manager.cc:1594] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:21.151572 21765 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:21.147864 21679 master_runner.cc:187] Error getting master registration for 127.20.202.254:43533: OK, instance_id { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" instance_seqno: 1786002860112731 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:21.153707 21679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:21.234102 21604 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 4
I20260806 07:54:21.245921 21604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Bootstrap starting.
I20260806 07:54:21.250607 21604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:21.252030 21604 log.cc:826] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:21.256392 21604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: No bootstrap required, opened a new log
I20260806 07:54:21.264547 21604 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.264937 21604 raft_consensus.cc:385] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:21.265082 21604 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec199fe4744d4e43818dcd65b7601095, State: Initialized, Role: FOLLOWER
I20260806 07:54:21.265877 21604 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.267010 21768 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.267292 21768 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.268430 21604 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:21.279933 21779 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:21.280135 21779 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:21.279212 21538 master_runner.cc:187] Error getting master registration for 127.20.202.253:45099: OK, instance_id { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" instance_seqno: 1786002860590128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:21.286880 21538 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:21.305666 21676 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } attempt: 5
W20260806 07:54:21.307345 21701 tablet.cc:2406] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:21.321375 21676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: Bootstrap starting.
I20260806 07:54:21.326205 21676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:21.327741 21676 log.cc:826] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:21.331213 21676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: No bootstrap required, opened a new log
I20260806 07:54:21.332082 21768 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:54:21.332335 21768 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.334184 21768 leader_election.cc:290] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f48d59f61bb48fe8e841a434e516672 (127.20.202.253:45099), cebfacc2d8b64fd299a42e9ed8865b80 (127.20.202.252:45113)
I20260806 07:54:21.336184 21722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec199fe4744d4e43818dcd65b7601095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" is_pre_election: true
I20260806 07:54:21.336771 21722 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec199fe4744d4e43818dcd65b7601095 in term 0.
I20260806 07:54:21.337675 21555 leader_election.cc:304] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cebfacc2d8b64fd299a42e9ed8865b80, ec199fe4744d4e43818dcd65b7601095; no voters:
I20260806 07:54:21.338299 21768 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:54:21.338452 21768 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:54:21.338551 21768 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:21.341223 21676 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.341635 21676 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:21.341797 21676 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f48d59f61bb48fe8e841a434e516672, State: Initialized, Role: FOLLOWER
I20260806 07:54:21.342571 21676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.344101 21768 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.347508 21722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec199fe4744d4e43818dcd65b7601095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cebfacc2d8b64fd299a42e9ed8865b80"
I20260806 07:54:21.347823 21722 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:54:21.349992 21558 tablet.cc:2406] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:21.355120 21722 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec199fe4744d4e43818dcd65b7601095 in term 1.
I20260806 07:54:21.356303 21555 leader_election.cc:304] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cebfacc2d8b64fd299a42e9ed8865b80, ec199fe4744d4e43818dcd65b7601095; no voters:
I20260806 07:54:21.360883 21768 leader_election.cc:290] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [CANDIDATE]: Term 1 election: Requested vote from peers 1f48d59f61bb48fe8e841a434e516672 (127.20.202.253:45099), cebfacc2d8b64fd299a42e9ed8865b80 (127.20.202.252:45113)
I20260806 07:54:21.362053 21768 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:54:21.362511 21768 raft_consensus.cc:697] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 LEADER]: Becoming Leader. State: Replica: ec199fe4744d4e43818dcd65b7601095, State: Running, Role: LEADER
I20260806 07:54:21.363788 21768 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } }
I20260806 07:54:21.373757 21676 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:21.373823 21649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec199fe4744d4e43818dcd65b7601095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f48d59f61bb48fe8e841a434e516672"
I20260806 07:54:21.374306 21649 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:21.373777 21650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec199fe4744d4e43818dcd65b7601095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f48d59f61bb48fe8e841a434e516672" is_pre_election: true
I20260806 07:54:21.381290 21649 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec199fe4744d4e43818dcd65b7601095 in term 1.
I20260806 07:54:21.373111 21782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.383172 21782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.389038 21783 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec199fe4744d4e43818dcd65b7601095. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.389477 21783 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:21.398867 21787 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:21.402117 21787 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:21.421693 21722 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Refusing update from remote peer ec199fe4744d4e43818dcd65b7601095: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:21.423024 21783 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:21.431610 21649 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Refusing update from remote peer ec199fe4744d4e43818dcd65b7601095: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:21.432849 21783 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:21.443192 21782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec199fe4744d4e43818dcd65b7601095. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.443475 21782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.449124 21790 mvcc.cc:204] Tried to move back new op lower bound from 7315467720364003328 to 7315467720160272384. Current Snapshot: MvccSnapshot[applied={T|T < 7315467720364003328}]
I20260806 07:54:21.461705 21754 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec199fe4744d4e43818dcd65b7601095. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.462074 21754 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.469970 21754 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.470279 21754 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.470605 21783 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.470861 21783 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:21.470890 21782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.471119 21782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:21.471145 21783 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } }
I20260806 07:54:21.471310 21783 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:21.474221 21787 catalog_manager.cc:1383] Generated new cluster ID: 935b6d902c6c4b58a73ec6d73217cb7a
I20260806 07:54:21.474344 21787 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:21.513814 21793 mvcc.cc:204] Tried to move back new op lower bound from 7315467720364003328 to 7315467720160272384. Current Snapshot: MvccSnapshot[applied={T|T < 7315467720364003328}]
I20260806 07:54:21.550028 21806 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 07:54:21.550581 21787 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:54:21.552438 21787 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:21.576673 21787 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Generated new TSK 0
I20260806 07:54:21.577885 21787 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:21.591342 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:54:21.996850 21807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:21.997215 21807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:21.997349 21807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:22.005966 21807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:22.006213 21807 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:22.016768 21807 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:22.018537 21807 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:22.020396 21807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:22.030400 21813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:22.030506 21815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:22.030435 21812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:22.032356 21807 server_base.cc:1061] running on GCE node
I20260806 07:54:22.032924 21807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:22.034147 21807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:22.035401 21807 hybrid_clock.cc:648] HybridClock initialized: now 1786002862035262 us; error 101 us; skew 500 ppm
I20260806 07:54:22.035758 21807 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:22.038013 21807 webserver.cc:533] Webserver started at http://127.20.202.193:44611/ using document root <none> and password file <none>
I20260806 07:54:22.038810 21807 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:22.038935 21807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:22.039304 21807 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:22.041719 21807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.042493 21807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.047801 21807 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:54:22.051270 21821 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:22.052603 21807 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:54:22.052789 21807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.053015 21807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:22.076807 21807 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:22.077888 21807 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:22.078190 21807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:22.079365 21807 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:22.081480 21807 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:22.081631 21807 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:22.081728 21807 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:22.081809 21807 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:22.118302 21807 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40219
I20260806 07:54:22.118384 21933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40219 every 8 connection(s)
I20260806 07:54:22.119874 21807 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:22.125207 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21807
I20260806 07:54:22.125440 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:54:22.154309 21765 catalog_manager.cc:1295] Loaded cluster ID: 935b6d902c6c4b58a73ec6d73217cb7a
I20260806 07:54:22.154541 21765 catalog_manager.cc:1588] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: loading cluster ID for follower catalog manager: success
I20260806 07:54:22.158869 21936 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45099
I20260806 07:54:22.159320 21936 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:22.159636 21935 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43533
I20260806 07:54:22.159787 21935 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:22.160427 21936 heartbeater.cc:507] Master 127.20.202.253:45099 requested a full tablet report, sending...
I20260806 07:54:22.160458 21935 heartbeater.cc:507] Master 127.20.202.254:43533 requested a full tablet report, sending...
I20260806 07:54:22.162197 21765 catalog_manager.cc:1610] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: acquiring CA information for follower catalog manager: success
I20260806 07:54:22.162954 21569 ts_manager.cc:194] Registered new tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:22.164906 21765 catalog_manager.cc:1638] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:22.165140 21569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:36091
I20260806 07:54:22.167781 21640 ts_manager.cc:194] Registered new tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:22.169044 21934 heartbeater.cc:344] Connected to a master server at 127.20.202.252:45113
I20260806 07:54:22.169189 21934 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:22.169509 21934 heartbeater.cc:507] Master 127.20.202.252:45113 requested a full tablet report, sending...
I20260806 07:54:22.171022 21712 ts_manager.cc:194] Registered new tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:22.182113 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:22.184413 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:35619
--webserver_interface=127.20.202.251
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619 with env {}
W20260806 07:54:22.534301 21944 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:22.534626 21944 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:22.534706 21944 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:22.543450 21944 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:22.543545 21944 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:22.543601 21944 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:22.543649 21944 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:22.554315 21944 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:35619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:22.555478 21806 catalog_manager.cc:1295] Loaded cluster ID: 935b6d902c6c4b58a73ec6d73217cb7a
I20260806 07:54:22.555608 21806 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: loading cluster ID for follower catalog manager: success
I20260806 07:54:22.556054 21944 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:22.558069 21944 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:54:22.559754 21806 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: acquiring CA information for follower catalog manager: success
I20260806 07:54:22.562264 21806 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:54:22.566095 21950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:22.566725 21951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:22.571134 21953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:22.571655 21944 server_base.cc:1061] running on GCE node
I20260806 07:54:22.572371 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:22.573987 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:22.575495 21944 hybrid_clock.cc:648] HybridClock initialized: now 1786002862575318 us; error 145 us; skew 500 ppm
I20260806 07:54:22.575953 21944 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:22.578509 21944 webserver.cc:533] Webserver started at http://127.20.202.251:42677/ using document root <none> and password file <none>
I20260806 07:54:22.579327 21944 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:22.579418 21944 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:22.579799 21944 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:22.582665 21944 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "7b6f7c58264e43e58d723581bb113dc1"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.583675 21944 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "7b6f7c58264e43e58d723581bb113dc1"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.589805 21944 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 07:54:22.593410 21959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:22.595050 21944 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:22.595510 21944 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "7b6f7c58264e43e58d723581bb113dc1"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.595796 21944 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:22.618395 21944 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:22.619408 21944 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:22.619724 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:22.647019 21944 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:35619
I20260806 07:54:22.647001 22010 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:35619 every 8 connection(s)
I20260806 07:54:22.648625 21944 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:22.653565 22011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:22.655309 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 21944
I20260806 07:54:22.663985 22011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:22.685866 22011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:22.692916 22011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:22.699709 22011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:22.710705 22011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Bootstrap starting.
I20260806 07:54:22.713927 22011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:22.715071 22011 log.cc:826] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:22.717821 22011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: No bootstrap required, opened a new log
I20260806 07:54:22.722882 22011 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } }
I20260806 07:54:22.723228 22011 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:22.723343 22011 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6f7c58264e43e58d723581bb113dc1, State: Initialized, Role: FOLLOWER
I20260806 07:54:22.724046 22011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } }
I20260806 07:54:22.725069 22020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } } }
I20260806 07:54:22.725430 22020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:22.726191 22011 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:22.738375 22031 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:22.738637 22031 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:22.746194 21944 master_runner.cc:429] Detected that this master 7b6f7c58264e43e58d723581bb113dc1 is joining an existing cluster
I20260806 07:54:22.746326 21944 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:54:22.747319 21944 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:54:22.747705 21944 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:54:22.747959 21944 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: stopping tablet replica
I20260806 07:54:22.749686 21944 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:35619
I20260806 07:54:22.804313 21569 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:35619: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 35619
}
attrs {
promote: true
}
}
dest_uuid: "ec199fe4744d4e43818dcd65b7601095"
cas_config_opid_index: -1
I20260806 07:54:22.805842 21569 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:22.809389 21649 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Refusing update from remote peer ec199fe4744d4e43818dcd65b7601095: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:22.809748 21722 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Refusing update from remote peer ec199fe4744d4e43818dcd65b7601095: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:22.810258 21799 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260806 07:54:22.811164 21553 consensus_peers.cc:597] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 -> Peer 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251:35619): Couldn't send request to peer 7b6f7c58264e43e58d723581bb113dc1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:54:22.811620 21799 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:22.825088 21799 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.833807 21721 raft_consensus.cc:2955] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.850136 22046 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.850503 22046 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:22.851897 21799 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:35619
I20260806 07:54:22.853045 21944 master.cc:562] Master@127.20.202.251:35619 shutting down...
I20260806 07:54:22.855988 22042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.856307 22042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:22.858155 21799 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.858410 21799 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:22.868268 21649 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.869475 22047 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.869836 22047 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:22.874089 22049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:22.874652 22049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:22.888823 21944 master.cc:584] Master@127.20.202.251:35619 shutdown complete.
I20260806 07:54:22.889016 21944 master_runner.cc:306] Clearing existing system tablet
I20260806 07:54:22.889532 21944 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:22.889739 21944 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:22.892747 21944 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:54:22.894650 22051 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:22.895107 21944 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:22.895504 21944 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 07:54:22.895673 21944 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "7b6f7c58264e43e58d723581bb113dc1"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.900434 21944 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:54:22.907884 21944 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:54:22.908128 21944 log.cc:1199] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:54:22.908753 21944 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Deleting consensus metadata
I20260806 07:54:22.909071 21944 master_runner.cc:316] Copying system tablet from 127.20.202.254:43533
I20260806 07:54:22.912127 21944 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.254:43533
I20260806 07:54:22.922552 21589 tablet_copy_service.cc:140] P ec199fe4744d4e43818dcd65b7601095: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7b6f7c58264e43e58d723581bb113dc1 ({username='slave'} at 127.0.0.1:59644)
I20260806 07:54:22.922888 21589 tablet_copy_service.cc:161] P ec199fe4744d4e43818dcd65b7601095: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7b6f7c58264e43e58d723581bb113dc1 at {username='slave'} at 127.0.0.1:59644: session id = 7b6f7c58264e43e58d723581bb113dc1-00000000000000000000000000000000
I20260806 07:54:22.927330 21589 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:54:22.929979 21944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:54:22.937294 21944 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: tablet copy: Starting download of 0 data blocks...
I20260806 07:54:22.937671 21944 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: tablet copy: Starting download of 1 WAL segments...
I20260806 07:54:22.941080 21944 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:54:22.946048 21589 tablet_copy_service.cc:342] P ec199fe4744d4e43818dcd65b7601095: Request end of tablet copy session 7b6f7c58264e43e58d723581bb113dc1-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:59644
I20260806 07:54:22.946319 21589 tablet_copy_service.cc:434] P ec199fe4744d4e43818dcd65b7601095: ending tablet copy session 7b6f7c58264e43e58d723581bb113dc1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7b6f7c58264e43e58d723581bb113dc1
W20260806 07:54:22.954826 21944 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:54:22.958520 21944 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:22.961225 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:22.966822 22059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:22.973640 22061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:22.973989 21944 server_base.cc:1061] running on GCE node
I20260806 07:54:22.974421 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:22.975394 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:22.976612 21944 hybrid_clock.cc:648] HybridClock initialized: now 1786002862976452 us; error 89 us; skew 500 ppm
I20260806 07:54:22.976830 21944 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:22.978304 21944 webserver.cc:533] Webserver started at http://127.20.202.251:32949/ using document root <none> and password file <none>
I20260806 07:54:22.978645 21944 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:22.978756 21944 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:22.981021 21944 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 07:54:22.982775 22067 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:22.983224 21944 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 07:54:22.983358 21944 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "7b6f7c58264e43e58d723581bb113dc1"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:22.983556 21944 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:23.002146 21944 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:23.002871 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:23.024436 21944 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:35619
I20260806 07:54:23.024457 22118 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:35619 every 8 connection(s)
I20260806 07:54:23.025508 21944 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:23.031950 22119 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:23.036804 22119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Bootstrap starting.
I20260806 07:54:23.064040 22119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:23.064739 22119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: Bootstrap complete.
I20260806 07:54:23.066510 22119 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:23.066905 22119 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7b6f7c58264e43e58d723581bb113dc1, State: Initialized, Role: LEARNER
I20260806 07:54:23.067281 22119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:23.070254 22123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:23.070585 22123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:23.070735 22119 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:23.080164 22134 catalog_manager.cc:1295] Loaded cluster ID: 935b6d902c6c4b58a73ec6d73217cb7a
I20260806 07:54:23.080291 22134 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: loading cluster ID for follower catalog manager: success
I20260806 07:54:23.084074 22134 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: acquiring CA information for follower catalog manager: success
I20260806 07:54:23.086982 22134 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:23.121549 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21807
I20260806 07:54:23.133961 21929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:23.222395 21935 heartbeater.cc:499] Master 127.20.202.254:43533 was elected leader, sending a full tablet report...
I20260806 07:54:23.224902 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21807
I20260806 07:54:23.251694 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40219
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--webserver_port=44611
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619 with env {}
I20260806 07:54:23.313525 22094 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:54:23.314427 22123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec199fe4744d4e43818dcd65b7601095. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:23.314673 22123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:54:23.621894 22137 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:23.622210 22137 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:23.622397 22137 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:23.630853 22137 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:23.631047 22137 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:23.641610 22137 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40219
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44611
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:23.643385 22137 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:23.645288 22137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:23.655380 22143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:23.655444 22144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:23.656544 22146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:23.657248 22137 server_base.cc:1061] running on GCE node
I20260806 07:54:23.657889 22137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:23.658978 22137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:23.660219 22137 hybrid_clock.cc:648] HybridClock initialized: now 1786002863660081 us; error 101 us; skew 500 ppm
I20260806 07:54:23.660511 22137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:23.662688 22137 webserver.cc:533] Webserver started at http://127.20.202.193:44611/ using document root <none> and password file <none>
I20260806 07:54:23.663400 22137 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:23.663506 22137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:23.668884 22137 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:54:23.672014 22152 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:23.673532 22137 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 07:54:23.673765 22137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:23.674558 22137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:23.699007 22137 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:23.699918 22137 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:23.700229 22137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:23.701222 22137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:23.702999 22137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:23.703073 22137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:23.703205 22137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:23.703261 22137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:23.737710 22137 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40219
I20260806 07:54:23.737798 22264 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40219 every 8 connection(s)
I20260806 07:54:23.739253 22137 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:23.745647 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22137
I20260806 07:54:23.762809 22267 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43533
I20260806 07:54:23.763216 22265 heartbeater.cc:344] Connected to a master server at 127.20.202.251:35619
I20260806 07:54:23.763324 22267 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:23.763475 22265 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:23.764799 22267 heartbeater.cc:507] Master 127.20.202.254:43533 requested a full tablet report, sending...
I20260806 07:54:23.764825 22265 heartbeater.cc:507] Master 127.20.202.251:35619 requested a full tablet report, sending...
I20260806 07:54:23.767217 21569 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:23.767575 22084 ts_manager.cc:194] Registered new tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:23.768208 21569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:49843
I20260806 07:54:23.771023 22266 heartbeater.cc:344] Connected to a master server at 127.20.202.252:45113
I20260806 07:54:23.771209 22266 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:23.771512 22268 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45099
I20260806 07:54:23.771708 22268 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:23.771719 22266 heartbeater.cc:507] Master 127.20.202.252:45113 requested a full tablet report, sending...
I20260806 07:54:23.772085 22268 heartbeater.cc:507] Master 127.20.202.253:45099 requested a full tablet report, sending...
I20260806 07:54:23.773484 21640 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:23.773772 21712 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:23.775039 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:23.784013 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:23.784165 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21538
I20260806 07:54:23.785099 21599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:23.902731 22042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec199fe4744d4e43818dcd65b7601095" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:23.903105 22042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:23.903913 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21538
W20260806 07:54:23.929298 21696 connection.cc:573] server connection from 127.0.0.1:49890 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:23.930415 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43533
--webserver_interface=127.20.202.254
--webserver_port=43069
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619 with env {}
W20260806 07:54:24.300415 22280 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:24.300751 22280 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:24.300822 22280 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:24.309442 22280 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:24.309543 22280 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:24.309617 22280 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:24.309693 22280 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:24.320019 22280 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=43069
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:24.321740 22280 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:24.323716 22280 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:24.332585 22286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:24.332667 22285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:24.334430 22288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:24.335436 22280 server_base.cc:1061] running on GCE node
I20260806 07:54:24.336313 22280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:24.338027 22280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:24.339241 22280 hybrid_clock.cc:648] HybridClock initialized: now 1786002864339098 us; error 105 us; skew 500 ppm
I20260806 07:54:24.339620 22280 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:24.341988 22280 webserver.cc:533] Webserver started at http://127.20.202.254:43069/ using document root <none> and password file <none>
I20260806 07:54:24.342794 22280 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:24.342907 22280 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:24.348769 22280 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:54:24.351800 22294 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:24.353312 22280 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:54:24.353502 22280 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "ec199fe4744d4e43818dcd65b7601095"
format_stamp: "Formatted at 2026-08-06 07:54:20 on dist-test-slave-g350"
I20260806 07:54:24.354326 22280 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:24.376861 22280 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:24.377893 22280 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:24.378213 22280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:24.400974 22280 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43533
I20260806 07:54:24.401006 22345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43533 every 8 connection(s)
I20260806 07:54:24.402830 22280 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:24.407212 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22280
I20260806 07:54:24.407557 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22137
I20260806 07:54:24.411573 22346 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:24.416458 22346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Bootstrap starting.
I20260806 07:54:24.417284 22260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:24.442540 22346 log.cc:826] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:24.454382 22346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:24.455073 22346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: Bootstrap complete.
I20260806 07:54:24.464054 22346 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:24.464775 22346 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec199fe4744d4e43818dcd65b7601095, State: Initialized, Role: FOLLOWER
I20260806 07:54:24.465579 22346 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:24.466593 22352 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:24.466917 22352 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:24.467756 22346 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:24.489451 22363 catalog_manager.cc:1295] Loaded cluster ID: 935b6d902c6c4b58a73ec6d73217cb7a
I20260806 07:54:24.489585 22363 catalog_manager.cc:1588] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: loading cluster ID for follower catalog manager: success
I20260806 07:54:24.493877 22363 catalog_manager.cc:1610] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: acquiring CA information for follower catalog manager: success
I20260806 07:54:24.496757 22363 catalog_manager.cc:1638] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:24.538275 22280 master_runner.cc:264] No leader master found from master ec199fe4744d4e43818dcd65b7601095
I20260806 07:54:24.538517 22280 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:24.541674 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22137
I20260806 07:54:24.561421 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40219
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--webserver_port=44611
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619 with env {}
W20260806 07:54:24.946545 22370 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:24.946911 22370 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:24.947052 22370 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:24.956228 22370 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:24.956455 22370 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:24.966980 22370 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40219
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44611
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:24.967779 22374 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec199fe4744d4e43818dcd65b7601095)
I20260806 07:54:24.968058 22374 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:24.969012 22370 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:24.969731 22374 leader_election.cc:290] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec199fe4744d4e43818dcd65b7601095 (127.20.202.254:43533), cebfacc2d8b64fd299a42e9ed8865b80 (127.20.202.252:45113)
I20260806 07:54:24.971100 21721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f48d59f61bb48fe8e841a434e516672" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" is_pre_election: true
I20260806 07:54:24.971513 22370 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:54:24.978170 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f48d59f61bb48fe8e841a434e516672" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "ec199fe4744d4e43818dcd65b7601095" is_pre_election: true
I20260806 07:54:24.978641 22321 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f48d59f61bb48fe8e841a434e516672 in term 1.
I20260806 07:54:24.979252 21626 leader_election.cc:304] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f48d59f61bb48fe8e841a434e516672, ec199fe4744d4e43818dcd65b7601095; no voters: cebfacc2d8b64fd299a42e9ed8865b80
I20260806 07:54:24.979722 22374 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:54:24.979959 22374 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec199fe4744d4e43818dcd65b7601095)
I20260806 07:54:24.980137 22374 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 1 FOLLOWER]: Advancing to term 2
W20260806 07:54:24.982609 22377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:24.983583 22378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:24.984645 22374 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:24.986268 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f48d59f61bb48fe8e841a434e516672" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "ec199fe4744d4e43818dcd65b7601095"
I20260806 07:54:24.986537 21721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f48d59f61bb48fe8e841a434e516672" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "cebfacc2d8b64fd299a42e9ed8865b80"
I20260806 07:54:24.986598 22321 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:54:24.985741 22374 leader_election.cc:290] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [CANDIDATE]: Term 2 election: Requested vote from peers ec199fe4744d4e43818dcd65b7601095 (127.20.202.254:43533), cebfacc2d8b64fd299a42e9ed8865b80 (127.20.202.252:45113)
W20260806 07:54:24.989997 22380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:24.990687 22370 server_base.cc:1061] running on GCE node
I20260806 07:54:24.991395 22370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:24.991839 22321 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f48d59f61bb48fe8e841a434e516672 in term 2.
I20260806 07:54:24.992368 21626 leader_election.cc:304] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f48d59f61bb48fe8e841a434e516672, ec199fe4744d4e43818dcd65b7601095; no voters: cebfacc2d8b64fd299a42e9ed8865b80
I20260806 07:54:24.992750 22374 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:54:24.993031 22370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:24.993237 22374 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 2 LEADER]: Becoming Leader. State: Replica: 1f48d59f61bb48fe8e841a434e516672, State: Running, Role: LEADER
I20260806 07:54:24.993793 22374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:24.995797 22383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f48d59f61bb48fe8e841a434e516672. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:24.996061 22383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:24.996444 22370 hybrid_clock.cc:648] HybridClock initialized: now 1786002864996211 us; error 188 us; skew 500 ppm
I20260806 07:54:24.996862 22370 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:24.996910 22385 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:24.999742 22385 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:24.999796 22370 webserver.cc:533] Webserver started at http://127.20.202.193:44611/ using document root <none> and password file <none>
I20260806 07:54:25.000846 22385 catalog_manager.cc:1295] Loaded cluster ID: 935b6d902c6c4b58a73ec6d73217cb7a
I20260806 07:54:25.000873 22370 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:25.000944 22385 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:25.001004 22370 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:25.002511 22385 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:25.003566 22385 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: Loaded TSK: 0
I20260806 07:54:25.004675 22385 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:25.007899 22370 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:54:25.010829 22389 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:25.012373 22370 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 07:54:25.012557 22370 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:25.013341 22370 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:25.051240 22370 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:25.052246 22370 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:25.052505 22370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:25.053654 22370 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:25.055723 22370 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:25.055830 22370 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:25.055974 22370 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:25.056047 22370 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:25.092417 22370 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40219
I20260806 07:54:25.092519 22501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40219 every 8 connection(s)
I20260806 07:54:25.094102 22370 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:25.102962 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22370
I20260806 07:54:25.103348 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22370
I20260806 07:54:25.123003 22505 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45099
I20260806 07:54:25.123462 22505 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:25.127643 22505 heartbeater.cc:507] Master 127.20.202.253:45099 requested a full tablet report, sending...
I20260806 07:54:25.128656 22503 heartbeater.cc:344] Connected to a master server at 127.20.202.252:45113
I20260806 07:54:25.128814 22503 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:25.129208 22502 heartbeater.cc:344] Connected to a master server at 127.20.202.251:35619
I20260806 07:54:25.129354 22502 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:25.129415 22503 heartbeater.cc:507] Master 127.20.202.252:45113 requested a full tablet report, sending...
I20260806 07:54:25.129971 22502 heartbeater.cc:507] Master 127.20.202.251:35619 requested a full tablet report, sending...
I20260806 07:54:25.130198 22504 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43533
I20260806 07:54:25.130348 22504 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:25.130856 22504 heartbeater.cc:507] Master 127.20.202.254:43533 requested a full tablet report, sending...
I20260806 07:54:25.130954 21640 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:25.130960 21712 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:25.132754 22311 ts_manager.cc:194] Registered new tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:25.133249 21640 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:46757
I20260806 07:54:25.133731 22084 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:25.137549 22497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:25.225155 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22370
I20260806 07:54:25.245138 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40219
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--webserver_port=44611
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--fault_before_start=1.0 with env {}
I20260806 07:54:25.405254 22517 raft_consensus.cc:493] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec199fe4744d4e43818dcd65b7601095)
I20260806 07:54:25.405516 22517 raft_consensus.cc:515] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } }
I20260806 07:54:25.407056 22517 leader_election.cc:290] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec199fe4744d4e43818dcd65b7601095 (127.20.202.254:43533), 1f48d59f61bb48fe8e841a434e516672 (127.20.202.253:45099)
I20260806 07:54:25.408756 21649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "1f48d59f61bb48fe8e841a434e516672" is_pre_election: true
I20260806 07:54:25.415160 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "ec199fe4744d4e43818dcd65b7601095" is_pre_election: true
I20260806 07:54:25.415534 22321 raft_consensus.cc:2393] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cebfacc2d8b64fd299a42e9ed8865b80 in current term 2: Already voted for candidate 1f48d59f61bb48fe8e841a434e516672 in this term.
I20260806 07:54:25.416333 21698 leader_election.cc:304] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cebfacc2d8b64fd299a42e9ed8865b80; no voters: 1f48d59f61bb48fe8e841a434e516672, ec199fe4744d4e43818dcd65b7601095
I20260806 07:54:25.416786 22517 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:54:25.420397 22517 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:54:25.438448 22321 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 2 FOLLOWER]: Refusing update from remote peer 1f48d59f61bb48fe8e841a434e516672: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:54:25.439381 22383 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:25.443730 22519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f48d59f61bb48fe8e841a434e516672. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.444110 22519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:25.445926 21721 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 2 FOLLOWER]: Refusing update from remote peer 1f48d59f61bb48fe8e841a434e516672: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:54:25.446681 22383 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:25.454602 22094 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 1 LEARNER]: Advancing to term 2
I20260806 07:54:25.459501 22094 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 2 LEARNER]: Refusing update from remote peer 1f48d59f61bb48fe8e841a434e516672: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:54:25.462503 22383 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:54:25.463629 22519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.463846 22519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:25.474788 22525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f48d59f61bb48fe8e841a434e516672. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.475188 22525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:25.474323 22528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.475580 22528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:25.476118 22522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.476490 22522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:25.476933 22374 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672: attempting to promote NON_VOTER 7b6f7c58264e43e58d723581bb113dc1 to VOTER
I20260806 07:54:25.478178 22374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } }
I20260806 07:54:25.480541 22093 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 2 LEARNER]: Refusing update from remote peer 1f48d59f61bb48fe8e841a434e516672: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:54:25.481210 22321 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 2 FOLLOWER]: Refusing update from remote peer 1f48d59f61bb48fe8e841a434e516672: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:54:25.481245 22522 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:54:25.482241 22522 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:54:25.485407 22517 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f48d59f61bb48fe8e841a434e516672. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.485144 22383 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.485785 22517 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:25.486398 21722 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 2 FOLLOWER]: Refusing update from remote peer 1f48d59f61bb48fe8e841a434e516672: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:54:25.487380 22522 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:54:25.486996 22321 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.488833 21722 raft_consensus.cc:2955] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.491554 22519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.491788 22519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec199fe4744d4e43818dcd65b7601095 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:25.485368 22530 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: true } } }
I20260806 07:54:25.492676 22530 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:25.493453 22522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.493705 22522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:25.493100 22093 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 7b6f7c58264e43e58d723581bb113dc1 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.494014 22522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.494196 22522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:25.494549 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.494740 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:25.495235 22530 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.495539 22530 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebfacc2d8b64fd299a42e9ed8865b80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:25.496949 22528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.497160 22528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6f7c58264e43e58d723581bb113dc1 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:54:25.620929 22516 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:25.621215 22516 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:25.621294 22516 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:54:25.621353 22516 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:25.629902 22516 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:25.630074 22516 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:25.640403 22516 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40219
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44611
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:25.642297 22516 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:25.644196 22516 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:25.654543 22536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:25.654570 22537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:25.654866 22539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:25.656867 22516 server_base.cc:1061] running on GCE node
I20260806 07:54:25.657442 22516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:25.658717 22516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:25.662083 22516 hybrid_clock.cc:648] HybridClock initialized: now 1786002865661893 us; error 125 us; skew 500 ppm
I20260806 07:54:25.662453 22516 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:25.664814 22516 webserver.cc:533] Webserver started at http://127.20.202.193:44611/ using document root <none> and password file <none>
I20260806 07:54:25.665709 22516 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:25.665859 22516 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:25.671468 22516 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:54:25.674360 22545 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:25.675968 22516 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:25.676151 22516 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:25.676893 22516 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:25.708806 22516 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:25.709843 22516 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:25.710135 22516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:25.711145 22516 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:25.712814 22516 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:25.712937 22516 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:25.713083 22516 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:25.713182 22516 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:54:25.713579 22516 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:54:25.713707 22516 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:54:25.713799 22516 fault_injection.cc:59] injecting fault for kudu (pid 22516): FLAGS_fault_before_start (process will exit)
I20260806 07:54:25.737686 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40219
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113
--webserver_port=44611
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619 with env {}
I20260806 07:54:25.885020 22531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1f48d59f61bb48fe8e841a434e516672" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ec199fe4744d4e43818dcd65b7601095" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43533 } } peers { permanent_uuid: "1f48d59f61bb48fe8e841a434e516672" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45099 } } peers { permanent_uuid: "cebfacc2d8b64fd299a42e9ed8865b80" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 45113 } } peers { permanent_uuid: "7b6f7c58264e43e58d723581bb113dc1" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 35619 } attrs { promote: false } } }
I20260806 07:54:25.885351 22531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f48d59f61bb48fe8e841a434e516672 [sys.catalog]: This master's current role is: LEADER
W20260806 07:54:26.102617 22553 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:26.103008 22553 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:26.103155 22553 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:26.111852 22553 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:26.112088 22553 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:26.122335 22553 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35187,127.20.202.213:45127,127.20.202.214:43867,127.20.202.215:39233,127.20.202.216:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40219
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=44611
--tserver_master_addrs=127.20.202.254:43533,127.20.202.253:45099,127.20.202.252:45113,127.20.202.251:35619
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:26.124114 22553 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:26.126154 22553 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:26.138653 22559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:26.138653 22561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:26.139521 22558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:26.139935 22553 server_base.cc:1061] running on GCE node
I20260806 07:54:26.140636 22553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:54:26.142036 22553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:54:26.143401 22553 hybrid_clock.cc:648] HybridClock initialized: now 1786002866143213 us; error 151 us; skew 500 ppm
I20260806 07:54:26.143734 22553 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:26.146250 22553 webserver.cc:533] Webserver started at http://127.20.202.193:44611/ using document root <none> and password file <none>
I20260806 07:54:26.147121 22553 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:26.147236 22553 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:26.153595 22553 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:54:26.157644 22567 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:26.159757 22553 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:54:26.159965 22553 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c6b50d7f83a445ea84b6c86012d77e9d"
format_stamp: "Formatted at 2026-08-06 07:54:22 on dist-test-slave-g350"
I20260806 07:54:26.160951 22553 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:26.177122 22553 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:26.178050 22553 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:26.178297 22553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:26.179286 22553 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:26.181087 22553 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:26.181164 22553 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:26.181334 22553 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:26.181391 22553 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:26.219000 22553 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40219
I20260806 07:54:26.219038 22679 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40219 every 8 connection(s)
I20260806 07:54:26.220777 22553 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:26.230218 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22553
I20260806 07:54:26.230686 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22553
I20260806 07:54:26.251075 22681 heartbeater.cc:344] Connected to a master server at 127.20.202.252:45113
I20260806 07:54:26.251219 22680 heartbeater.cc:344] Connected to a master server at 127.20.202.251:35619
I20260806 07:54:26.251547 22682 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43533
I20260806 07:54:26.251587 22681 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:26.251741 22682 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:26.251590 22680 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:26.253443 22683 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45099
I20260806 07:54:26.253659 22682 heartbeater.cc:507] Master 127.20.202.254:43533 requested a full tablet report, sending...
I20260806 07:54:26.253875 22681 heartbeater.cc:507] Master 127.20.202.252:45113 requested a full tablet report, sending...
I20260806 07:54:26.253674 22680 heartbeater.cc:507] Master 127.20.202.251:35619 requested a full tablet report, sending...
I20260806 07:54:26.253679 22683 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:26.255373 22683 heartbeater.cc:507] Master 127.20.202.253:45099 requested a full tablet report, sending...
I20260806 07:54:26.256237 22311 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:26.256693 22084 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:26.257733 21640 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:26.257952 21712 ts_manager.cc:194] Re-registered known tserver with Master: c6b50d7f83a445ea84b6c86012d77e9d (127.20.202.193:40219)
I20260806 07:54:26.258893 21640 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:33959
I20260806 07:54:26.261698 22675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:26.352211 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22553
I20260806 07:54:26.371474 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22280
I20260806 07:54:26.378206 22341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:26.487421 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22280
I20260806 07:54:26.511217 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21608
I20260806 07:54:26.512369 21671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:26.625552 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21608
I20260806 07:54:26.649183 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21679
I20260806 07:54:26.650553 21742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:26.762887 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21679
I20260806 07:54:26.786229 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 21944
I20260806 07:54:26.787400 22114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:26.900859 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 21944
2026-08-06T07:54:26Z chronyd exiting
2026-08-06T07:54:26Z chronyd exiting
2026-08-06T07:54:26Z chronyd exiting
2026-08-06T07:54:26Z chronyd exiting
2026-08-06T07:54:26Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/2 (7590 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/3
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:54:34: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:54:43.107304 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36243
--webserver_interface=127.20.202.254
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357 with env {}
W20260806 07:54:43.464423 22977 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:43.464762 22977 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:43.464860 22977 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:43.473335 22977 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:43.473423 22977 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:43.473512 22977 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:43.473626 22977 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:43.483848 22977 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:43.485483 22977 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:43.487370 22977 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:43.495697 22982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:43.495745 22983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:43.496054 22985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:43.496758 22977 server_base.cc:1061] running on GCE node
I20260806 07:54:43.497529 22977 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:43.497740 22977 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:43.497852 22977 hybrid_clock.cc:648] HybridClock initialized: now 1786002883497807 us; error 0 us; skew 500 ppm
I20260806 07:54:43.498190 22977 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:43.500945 22977 webserver.cc:533] Webserver started at http://127.20.202.254:33311/ using document root <none> and password file <none>
I20260806 07:54:43.501766 22977 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:43.501888 22977 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:43.502278 22977 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:43.504639 22977 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "17346d8291dc4d74a9f90f7d0023248b"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:43.505443 22977 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "17346d8291dc4d74a9f90f7d0023248b"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:43.510866 22977 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 07:54:43.514288 22990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:43.515760 22977 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:54:43.515982 22977 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "17346d8291dc4d74a9f90f7d0023248b"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:43.516201 22977 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:43.541445 22977 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:43.542493 22977 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:43.542790 22977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:43.564164 22977 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:36243
I20260806 07:54:43.564191 23041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:36243 every 8 connection(s)
I20260806 07:54:43.565810 22977 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:43.570319 23042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:43.572033 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 22977
I20260806 07:54:43.572268 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:54:43.573681 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:46797
--webserver_interface=127.20.202.253
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357 with env {}
I20260806 07:54:43.579075 23042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:43.601061 23042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:43.602284 22991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:46797: connect: Connection refused (error 111)
W20260806 07:54:43.603929 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:46797: Network error: Client connection negotiation failed: client connection to 127.20.202.253:46797: connect: Connection refused (error 111)
I20260806 07:54:43.653319 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } attempt: 1
W20260806 07:54:43.655629 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:46797: Network error: Client connection negotiation failed: client connection to 127.20.202.253:46797: connect: Connection refused (error 111)
I20260806 07:54:43.723930 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } attempt: 2
W20260806 07:54:43.726164 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:46797: Network error: Client connection negotiation failed: client connection to 127.20.202.253:46797: connect: Connection refused (error 111)
I20260806 07:54:43.817431 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } attempt: 3
W20260806 07:54:43.819614 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:46797: Network error: Client connection negotiation failed: client connection to 127.20.202.253:46797: connect: Connection refused (error 111)
W20260806 07:54:43.926695 23046 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:43.927044 23046 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:43.927187 23046 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:43.935408 23046 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:43.935496 23046 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:43.935602 23046 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:43.935711 23046 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:43.948519 23046 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:46797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:43.950321 23046 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:43.952217 23046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:43.960716 23052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:43.960780 23053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:43.960780 23055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:43.962184 23046 server_base.cc:1061] running on GCE node
I20260806 07:54:43.962914 23046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260806 07:54:43.962862 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } attempt: 4
W20260806 07:54:43.963116 23046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:43.963217 23046 hybrid_clock.cc:648] HybridClock initialized: now 1786002883963195 us; error 0 us; skew 500 ppm
I20260806 07:54:43.963624 23046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260806 07:54:43.965104 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:46797: Network error: Client connection negotiation failed: client connection to 127.20.202.253:46797: connect: Connection refused (error 111)
I20260806 07:54:43.966699 23046 webserver.cc:533] Webserver started at http://127.20.202.253:41833/ using document root <none> and password file <none>
I20260806 07:54:43.967489 23046 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:43.967619 23046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:43.967995 23046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:43.970472 23046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "3979fcaad53144bd856a9b9a8bd86b41"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:43.971266 23046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "3979fcaad53144bd856a9b9a8bd86b41"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:43.976583 23046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:54:43.980017 23060 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:43.981297 23046 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 07:54:43.981480 23046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "3979fcaad53144bd856a9b9a8bd86b41"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:43.981767 23046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:44.003173 23046 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:44.004094 23046 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:44.004374 23046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:44.025381 23111 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:46797 every 8 connection(s)
I20260806 07:54:44.025405 23046 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:46797
I20260806 07:54:44.027079 23046 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:54:44.028586 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23046
I20260806 07:54:44.028784 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:54:44.030099 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:37357
--webserver_interface=127.20.202.252
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357 with env {}
I20260806 07:54:44.032824 23112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:44.042328 23112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:44.058089 23112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:44.071275 23112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:44.072638 23063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
W20260806 07:54:44.074604 23112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
I20260806 07:54:44.123981 23112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 1
W20260806 07:54:44.126430 23112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
I20260806 07:54:44.194694 23112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 2
W20260806 07:54:44.196808 23112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
I20260806 07:54:44.264387 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } attempt: 5
I20260806 07:54:44.270998 23042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:54:44.273120 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
I20260806 07:54:44.288028 23112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 3
W20260806 07:54:44.290181 23112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
I20260806 07:54:44.324375 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 1
W20260806 07:54:44.326663 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
W20260806 07:54:44.388216 23114 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:44.388644 23114 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:44.388725 23114 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:54:44.394927 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 2
W20260806 07:54:44.397099 23042 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
W20260806 07:54:44.397526 23114 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:44.397658 23114 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:44.397728 23114 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:44.397776 23114 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:44.407706 23114 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:37357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:44.409297 23114 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:44.411635 23114 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:44.420377 23122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:44.420404 23123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:44.420715 23125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:44.421908 23114 server_base.cc:1061] running on GCE node
I20260806 07:54:44.422693 23114 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:44.422844 23114 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:44.422967 23114 hybrid_clock.cc:648] HybridClock initialized: now 1786002884422938 us; error 0 us; skew 500 ppm
I20260806 07:54:44.423328 23114 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:44.426173 23114 webserver.cc:533] Webserver started at http://127.20.202.252:40361/ using document root <none> and password file <none>
I20260806 07:54:44.426901 23114 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:44.426996 23114 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:44.427302 23114 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:44.429770 23114 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "ebcfe9bfa65f40c78193bd52e01d83a2"
format_stamp: "Formatted at 2026-08-06 07:54:44 on dist-test-slave-g350"
I20260806 07:54:44.430552 23114 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "ebcfe9bfa65f40c78193bd52e01d83a2"
format_stamp: "Formatted at 2026-08-06 07:54:44 on dist-test-slave-g350"
I20260806 07:54:44.433401 23112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 4
W20260806 07:54:44.435473 23112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:37357: Network error: Client connection negotiation failed: client connection to 127.20.202.252:37357: connect: Connection refused (error 111)
I20260806 07:54:44.436478 23114 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:54:44.440428 23130 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:44.441955 23114 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:44.442155 23114 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "ebcfe9bfa65f40c78193bd52e01d83a2"
format_stamp: "Formatted at 2026-08-06 07:54:44 on dist-test-slave-g350"
I20260806 07:54:44.442391 23114 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:44.457535 23114 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:44.458527 23114 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:44.458822 23114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:44.479525 23114 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:37357
I20260806 07:54:44.479615 23181 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:37357 every 8 connection(s)
I20260806 07:54:44.481271 23114 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:54:44.485503 23182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:44.487540 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23114
I20260806 07:54:44.487766 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:54:44.492928 23182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:44.503345 23042 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 3
I20260806 07:54:44.508718 23182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:44.515514 23182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:44.518281 23042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Bootstrap starting.
I20260806 07:54:44.522706 23042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:44.524258 23042 log.cc:826] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:44.526211 23182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: Bootstrap starting.
I20260806 07:54:44.527318 23042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: No bootstrap required, opened a new log
I20260806 07:54:44.529244 23182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:44.530350 23182 log.cc:826] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:44.532876 23182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: No bootstrap required, opened a new log
I20260806 07:54:44.532850 23042 raft_consensus.cc:359] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.533170 23042 raft_consensus.cc:385] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:44.533298 23042 raft_consensus.cc:740] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17346d8291dc4d74a9f90f7d0023248b, State: Initialized, Role: FOLLOWER
I20260806 07:54:44.534103 23042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.535290 23191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.535609 23191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:44.536516 23042 sys_catalog.cc:565] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:44.539054 23182 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.539304 23182 raft_consensus.cc:385] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:44.539492 23182 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcfe9bfa65f40c78193bd52e01d83a2, State: Initialized, Role: FOLLOWER
I20260806 07:54:44.540073 23182 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.541155 23199 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.541517 23199 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:44.542143 23182 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:44.545655 22996 tablet.cc:2406] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:54:44.547806 23203 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:44.547926 23203 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:44.550583 22977 master_runner.cc:187] Error getting master registration for 127.20.202.253:46797: OK, instance_id { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" instance_seqno: 1786002884019592 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:44.559108 22977 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:44.559689 23114 master_runner.cc:187] Error getting master registration for 127.20.202.253:46797: OK, instance_id { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" instance_seqno: 1786002884019592 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:54:44.564443 23114 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:54:44.572155 23215 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:44.572367 23215 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:54:44.711020 23136 tablet.cc:2406] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:54:44.734737 23112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } attempt: 5
I20260806 07:54:44.752463 23112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: Bootstrap starting.
I20260806 07:54:44.756749 23112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:44.758137 23112 log.cc:826] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:44.762200 23112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: No bootstrap required, opened a new log
I20260806 07:54:44.771114 23112 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.771472 23112 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:44.771668 23112 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3979fcaad53144bd856a9b9a8bd86b41, State: Initialized, Role: FOLLOWER
I20260806 07:54:44.772410 23112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.773458 23218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.773860 23218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:44.774776 23112 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:44.792776 23046 master_runner.cc:264] No leader master found from master 3979fcaad53144bd856a9b9a8bd86b41
I20260806 07:54:44.792928 23046 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:54:44.795887 23229 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:44.796150 23229 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:44.830034 23199 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:54:44.830415 23199 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.838841 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17346d8291dc4d74a9f90f7d0023248b" is_pre_election: true
I20260806 07:54:44.839566 23017 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebcfe9bfa65f40c78193bd52e01d83a2 in term 0.
I20260806 07:54:44.850878 23199 leader_election.cc:290] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17346d8291dc4d74a9f90f7d0023248b (127.20.202.254:36243), 3979fcaad53144bd856a9b9a8bd86b41 (127.20.202.253:46797)
I20260806 07:54:44.852519 23131 leader_election.cc:304] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17346d8291dc4d74a9f90f7d0023248b, ebcfe9bfa65f40c78193bd52e01d83a2; no voters:
I20260806 07:54:44.853976 23087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3979fcaad53144bd856a9b9a8bd86b41" is_pre_election: true
I20260806 07:54:44.854624 23087 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebcfe9bfa65f40c78193bd52e01d83a2 in term 0.
I20260806 07:54:44.855870 23199 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:54:44.856132 23199 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:54:44.856330 23199 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:44.861330 23199 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.862331 23199 leader_election.cc:290] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [CANDIDATE]: Term 1 election: Requested vote from peers 17346d8291dc4d74a9f90f7d0023248b (127.20.202.254:36243), 3979fcaad53144bd856a9b9a8bd86b41 (127.20.202.253:46797)
I20260806 07:54:44.866631 23087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3979fcaad53144bd856a9b9a8bd86b41"
I20260806 07:54:44.866879 23087 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:44.874269 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17346d8291dc4d74a9f90f7d0023248b"
I20260806 07:54:44.874543 23017 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:54:44.884403 23087 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebcfe9bfa65f40c78193bd52e01d83a2 in term 1.
I20260806 07:54:44.890130 23131 leader_election.cc:304] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3979fcaad53144bd856a9b9a8bd86b41, ebcfe9bfa65f40c78193bd52e01d83a2; no voters:
I20260806 07:54:44.891129 23199 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:54:44.892473 23017 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebcfe9bfa65f40c78193bd52e01d83a2 in term 1.
I20260806 07:54:44.896252 23199 raft_consensus.cc:697] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 1 LEADER]: Becoming Leader. State: Replica: ebcfe9bfa65f40c78193bd52e01d83a2, State: Running, Role: LEADER
I20260806 07:54:44.897274 23199 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } }
I20260806 07:54:44.917785 23231 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcfe9bfa65f40c78193bd52e01d83a2. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.918222 23231 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:44.926399 23233 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:54:44.929168 23233 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:54:44.951190 23087 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 1 FOLLOWER]: Refusing update from remote peer ebcfe9bfa65f40c78193bd52e01d83a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:44.953048 23017 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Refusing update from remote peer ebcfe9bfa65f40c78193bd52e01d83a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:54:44.955224 23231 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:44.956892 23231 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:54:44.981135 23191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcfe9bfa65f40c78193bd52e01d83a2. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.981559 23191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:44.988962 23231 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.989236 23231 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:44.990852 23233 catalog_manager.cc:1383] Generated new cluster ID: fd73481fd533440d94dc57feb090c63f
I20260806 07:54:44.990993 23233 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:54:44.991312 23218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcfe9bfa65f40c78193bd52e01d83a2. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.991575 23218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:44.993019 23199 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:44.993242 23199 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:44.995285 23236 mvcc.cc:204] Tried to move back new op lower bound from 7315467816731791360 to 7315467816546304000. Current Snapshot: MvccSnapshot[applied={T|T < 7315467816731791360}]
I20260806 07:54:45.008493 23218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:45.010090 23218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:45.018251 23191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } }
I20260806 07:54:45.018527 23191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:45.053696 23233 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:54:45.055591 23233 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:54:45.085304 23233 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: Generated new TSK 0
I20260806 07:54:45.089080 23233 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:54:45.105463 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--time_source=system_unsync with env {}
W20260806 07:54:45.466651 23243 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:45.467041 23243 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:45.467185 23243 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:45.475477 23243 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:45.475714 23243 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:45.486078 23243 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:45.487923 23243 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:45.489804 23243 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:45.499457 23249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:45.499382 23248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:45.499505 23251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:45.501021 23243 server_base.cc:1061] running on GCE node
I20260806 07:54:45.501516 23243 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:45.501725 23243 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:45.501797 23243 hybrid_clock.cc:648] HybridClock initialized: now 1786002885501780 us; error 0 us; skew 500 ppm
I20260806 07:54:45.502049 23243 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:45.504176 23243 webserver.cc:533] Webserver started at http://127.20.202.193:35651/ using document root <none> and password file <none>
I20260806 07:54:45.504884 23243 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:45.504971 23243 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:45.505266 23243 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:45.507643 23243 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:45.508416 23243 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:45.513839 23243 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:54:45.517194 23256 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:45.518777 23243 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:45.518968 23243 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:45.519207 23243 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:45.547884 23243 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:45.548987 23243 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:45.549283 23243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:45.550001 23203 catalog_manager.cc:1295] Loaded cluster ID: fd73481fd533440d94dc57feb090c63f
I20260806 07:54:45.550132 23203 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: loading cluster ID for follower catalog manager: success
I20260806 07:54:45.550457 23243 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:45.552235 23243 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:45.552363 23243 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:45.552518 23243 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:45.552608 23243 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:45.553377 23203 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: acquiring CA information for follower catalog manager: success
I20260806 07:54:45.555228 23203 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:45.590401 23243 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42177
I20260806 07:54:45.590453 23368 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42177 every 8 connection(s)
I20260806 07:54:45.591831 23243 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:45.599978 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23243
I20260806 07:54:45.600158 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:54:45.612895 23371 heartbeater.cc:344] Connected to a master server at 127.20.202.253:46797
I20260806 07:54:45.613071 23369 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37357
I20260806 07:54:45.613348 23371 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:45.613355 23369 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:45.614657 23371 heartbeater.cc:507] Master 127.20.202.253:46797 requested a full tablet report, sending...
I20260806 07:54:45.614679 23369 heartbeater.cc:507] Master 127.20.202.252:37357 requested a full tablet report, sending...
I20260806 07:54:45.617383 23147 ts_manager.cc:194] Registered new tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:45.617439 23077 ts_manager.cc:194] Registered new tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:45.619442 23147 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:58693
I20260806 07:54:45.626958 23370 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36243
I20260806 07:54:45.627096 23370 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:45.627378 23370 heartbeater.cc:507] Master 127.20.202.254:36243 requested a full tablet report, sending...
I20260806 07:54:45.629065 23007 ts_manager.cc:194] Registered new tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:45.638612 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:45.640880 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:44865
--webserver_interface=127.20.202.251
--webserver_port=0
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865 with env {}
I20260806 07:54:45.798209 23229 catalog_manager.cc:1295] Loaded cluster ID: fd73481fd533440d94dc57feb090c63f
I20260806 07:54:45.798316 23229 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: loading cluster ID for follower catalog manager: success
I20260806 07:54:45.801159 23229 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: acquiring CA information for follower catalog manager: success
I20260806 07:54:45.802994 23229 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:54:45.996707 23380 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:45.997058 23380 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:45.997162 23380 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:46.005654 23380 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:46.005749 23380 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:46.005838 23380 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:46.005900 23380 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:46.016083 23380 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:46.017839 23380 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:46.019649 23380 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:46.027995 23386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:46.028692 23385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:46.029245 23388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:46.030644 23380 server_base.cc:1061] running on GCE node
I20260806 07:54:46.031530 23380 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:46.031745 23380 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:46.031845 23380 hybrid_clock.cc:648] HybridClock initialized: now 1786002886031824 us; error 0 us; skew 500 ppm
I20260806 07:54:46.032240 23380 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:46.035642 23380 webserver.cc:533] Webserver started at http://127.20.202.251:40159/ using document root <none> and password file <none>
I20260806 07:54:46.036615 23380 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:46.036732 23380 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:46.037156 23380 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:54:46.040386 23380 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "31cac06e11ed4fb7b755795a9217252b"
format_stamp: "Formatted at 2026-08-06 07:54:46 on dist-test-slave-g350"
I20260806 07:54:46.041375 23380 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "31cac06e11ed4fb7b755795a9217252b"
format_stamp: "Formatted at 2026-08-06 07:54:46 on dist-test-slave-g350"
I20260806 07:54:46.048297 23380 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:54:46.052304 23393 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:46.053983 23380 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:54:46.054183 23380 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "31cac06e11ed4fb7b755795a9217252b"
format_stamp: "Formatted at 2026-08-06 07:54:46 on dist-test-slave-g350"
I20260806 07:54:46.054406 23380 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:46.075968 23380 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:46.076995 23380 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:46.077309 23380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:46.098307 23380 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:44865
I20260806 07:54:46.098383 23444 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:44865 every 8 connection(s)
I20260806 07:54:46.099998 23380 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:46.101792 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23380
I20260806 07:54:46.105487 23445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:54:46.112123 23445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:46.122361 23445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:46.128851 23445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:46.134859 23445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:54:46.144647 23445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Bootstrap starting.
I20260806 07:54:46.147991 23445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Neither blocks nor log segments found. Creating new log.
I20260806 07:54:46.149175 23445 log.cc:826] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:46.151724 23445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: No bootstrap required, opened a new log
I20260806 07:54:46.156739 23445 raft_consensus.cc:359] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } }
I20260806 07:54:46.157055 23445 raft_consensus.cc:385] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:54:46.157172 23445 raft_consensus.cc:740] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31cac06e11ed4fb7b755795a9217252b, State: Initialized, Role: FOLLOWER
I20260806 07:54:46.157850 23445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } }
I20260806 07:54:46.158723 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } } }
I20260806 07:54:46.159027 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:46.159731 23445 sys_catalog.cc:565] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:54:46.170631 23464 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:54:46.170791 23464 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:54:46.172580 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23243
I20260806 07:54:46.182466 23380 master_runner.cc:429] Detected that this master 31cac06e11ed4fb7b755795a9217252b is joining an existing cluster
I20260806 07:54:46.182569 23380 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:54:46.183569 23380 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:54:46.183933 23380 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:54:46.184162 23380 tablet_replica.cc:333] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: stopping tablet replica
I20260806 07:54:46.184983 23364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:46.186120 23380 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:44865
I20260806 07:54:46.225121 23147 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:44865: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "31cac06e11ed4fb7b755795a9217252b"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 44865
}
attrs {
promote: true
}
}
dest_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2"
cas_config_opid_index: -1
I20260806 07:54:46.226824 23147 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } }
W20260806 07:54:46.230659 23134 consensus_peers.cc:597] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 -> Peer 31cac06e11ed4fb7b755795a9217252b (127.20.202.251:44865): Couldn't send request to peer 31cac06e11ed4fb7b755795a9217252b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:54:46.231930 23017 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Refusing update from remote peer ebcfe9bfa65f40c78193bd52e01d83a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:46.232252 23086 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 1 FOLLOWER]: Refusing update from remote peer ebcfe9bfa65f40c78193bd52e01d83a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:54:46.233186 23377 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:46.233489 23477 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:54:46.247361 23377 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.260560 23017 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.262764 23377 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:44865
I20260806 07:54:46.263638 23380 master.cc:562] Master@127.20.202.251:44865 shutting down...
I20260806 07:54:46.258682 23086 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.268837 23477 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.268916 23480 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.269250 23477 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:46.269315 23480 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:46.270884 23480 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.271050 23480 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:46.278959 23482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.279472 23482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:46.280107 23483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.280445 23483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:46.292884 23380 master.cc:584] Master@127.20.202.251:44865 shutdown complete.
I20260806 07:54:46.293005 23380 master_runner.cc:306] Clearing existing system tablet
I20260806 07:54:46.293399 23380 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:46.293516 23380 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:46.296424 23380 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:54:46.298148 23486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:46.298581 23380 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:46.298825 23380 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 07:54:46.298914 23380 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "31cac06e11ed4fb7b755795a9217252b"
format_stamp: "Formatted at 2026-08-06 07:54:46 on dist-test-slave-g350"
I20260806 07:54:46.303627 23380 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:54:46.310801 23380 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:54:46.310963 23380 log.cc:1199] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:54:46.311450 23380 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Deleting consensus metadata
I20260806 07:54:46.311691 23380 master_runner.cc:316] Copying system tablet from 127.20.202.252:37357
I20260806 07:54:46.314530 23380 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.252:37357
I20260806 07:54:46.324519 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23243
I20260806 07:54:46.333989 23167 tablet_copy_service.cc:140] P ebcfe9bfa65f40c78193bd52e01d83a2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 31cac06e11ed4fb7b755795a9217252b ({username='slave'} at 127.0.0.1:48964)
I20260806 07:54:46.334334 23167 tablet_copy_service.cc:161] P ebcfe9bfa65f40c78193bd52e01d83a2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 31cac06e11ed4fb7b755795a9217252b at {username='slave'} at 127.0.0.1:48964: session id = 31cac06e11ed4fb7b755795a9217252b-00000000000000000000000000000000
I20260806 07:54:46.339473 23167 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:54:46.342309 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:54:46.348519 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42177
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--webserver_port=35651
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865 with env {}
I20260806 07:54:46.349948 23380 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: tablet copy: Starting download of 0 data blocks...
I20260806 07:54:46.350258 23380 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: tablet copy: Starting download of 1 WAL segments...
I20260806 07:54:46.353658 23380 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:54:46.358546 23167 tablet_copy_service.cc:342] P ebcfe9bfa65f40c78193bd52e01d83a2: Request end of tablet copy session 31cac06e11ed4fb7b755795a9217252b-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:48964
I20260806 07:54:46.358805 23167 tablet_copy_service.cc:434] P ebcfe9bfa65f40c78193bd52e01d83a2: ending tablet copy session 31cac06e11ed4fb7b755795a9217252b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 31cac06e11ed4fb7b755795a9217252b
I20260806 07:54:46.371510 23380 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:46.375152 23494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:46.375454 23495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:46.383119 23497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:46.388047 23380 server_base.cc:1061] running on GCE node
I20260806 07:54:46.388422 23380 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:46.388532 23380 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:46.388622 23380 hybrid_clock.cc:648] HybridClock initialized: now 1786002886388598 us; error 0 us; skew 500 ppm
I20260806 07:54:46.388842 23380 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:46.390275 23380 webserver.cc:533] Webserver started at http://127.20.202.251:33785/ using document root <none> and password file <none>
I20260806 07:54:46.390700 23380 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:46.390789 23380 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:46.393770 23380 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 07:54:46.395641 23502 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:46.396078 23380 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 07:54:46.396214 23380 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "31cac06e11ed4fb7b755795a9217252b"
format_stamp: "Formatted at 2026-08-06 07:54:46 on dist-test-slave-g350"
I20260806 07:54:46.396353 23380 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:46.436403 23380 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:46.437323 23380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:46.460815 23380 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:44865
I20260806 07:54:46.460845 23553 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:44865 every 8 connection(s)
I20260806 07:54:46.462076 23380 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:54:46.467239 23554 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:46.471055 23554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Bootstrap starting.
I20260806 07:54:46.486500 23554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:46.487035 23554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: Bootstrap complete.
I20260806 07:54:46.488113 23554 raft_consensus.cc:359] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } }
I20260806 07:54:46.488559 23554 raft_consensus.cc:740] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 31cac06e11ed4fb7b755795a9217252b, State: Initialized, Role: LEARNER
I20260806 07:54:46.488798 23554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } }
I20260806 07:54:46.502135 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.502425 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: This master's current role is: LEARNER
I20260806 07:54:46.502462 23554 sys_catalog.cc:565] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:46.515429 23568 catalog_manager.cc:1295] Loaded cluster ID: fd73481fd533440d94dc57feb090c63f
I20260806 07:54:46.515564 23568 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: loading cluster ID for follower catalog manager: success
I20260806 07:54:46.518224 23568 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: acquiring CA information for follower catalog manager: success
I20260806 07:54:46.520031 23568 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:46.663429 23529 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:54:46.664327 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcfe9bfa65f40c78193bd52e01d83a2. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:46.664604 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: This master's current role is: LEARNER
W20260806 07:54:46.786101 23493 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:46.786509 23493 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:46.786655 23493 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:46.795245 23493 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:46.795457 23493 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:46.805861 23493 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42177
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35651
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:46.807593 23493 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:46.809630 23493 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:46.819725 23574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:46.819703 23575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:46.821092 23493 server_base.cc:1061] running on GCE node
W20260806 07:54:46.821267 23577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:46.821941 23493 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:46.822154 23493 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:46.822270 23493 hybrid_clock.cc:648] HybridClock initialized: now 1786002886822245 us; error 0 us; skew 500 ppm
I20260806 07:54:46.822602 23493 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:46.824915 23493 webserver.cc:533] Webserver started at http://127.20.202.193:35651/ using document root <none> and password file <none>
I20260806 07:54:46.825778 23493 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:46.825917 23493 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:46.832018 23493 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:54:46.834962 23582 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:46.836434 23493 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:46.836573 23493 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:46.837324 23493 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:46.852689 23493 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:46.853526 23493 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:46.853863 23493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:46.854797 23493 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:46.856546 23493 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:46.856619 23493 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:46.856880 23493 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:46.857004 23493 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:46.896140 23493 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42177
I20260806 07:54:46.896190 23694 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42177 every 8 connection(s)
I20260806 07:54:46.898090 23493 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:46.900228 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23493
I20260806 07:54:46.931391 23695 heartbeater.cc:344] Connected to a master server at 127.20.202.251:44865
I20260806 07:54:46.931943 23695 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:46.933274 23695 heartbeater.cc:507] Master 127.20.202.251:44865 requested a full tablet report, sending...
I20260806 07:54:46.935592 23696 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37357
I20260806 07:54:46.935801 23696 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:46.936312 23519 ts_manager.cc:194] Registered new tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:46.936492 23696 heartbeater.cc:507] Master 127.20.202.252:37357 requested a full tablet report, sending...
I20260806 07:54:46.938292 23697 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36243
I20260806 07:54:46.938356 23147 ts_manager.cc:194] Re-registered known tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:46.938463 23697 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:46.938997 23697 heartbeater.cc:507] Master 127.20.202.254:36243 requested a full tablet report, sending...
I20260806 07:54:46.940083 23698 heartbeater.cc:344] Connected to a master server at 127.20.202.253:46797
I20260806 07:54:46.940234 23698 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:46.940282 23147 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:55679
I20260806 07:54:46.940574 23007 ts_manager.cc:194] Re-registered known tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:46.940776 23698 heartbeater.cc:507] Master 127.20.202.253:46797 requested a full tablet report, sending...
I20260806 07:54:46.942152 23077 ts_manager.cc:194] Re-registered known tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:46.943442 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:46.952154 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:54:46.952309 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 22977
I20260806 07:54:46.953212 23037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:47.069006 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 22977
I20260806 07:54:47.092914 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36243
--webserver_interface=127.20.202.254
--webserver_port=33311
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865 with env {}
I20260806 07:54:47.258327 23477 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:47.258767 23477 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:47.260483 23711 raft_consensus.cc:1064] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2: attempting to promote NON_VOTER 31cac06e11ed4fb7b755795a9217252b to VOTER
I20260806 07:54:47.261287 23711 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } }
I20260806 07:54:47.264308 23529 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 1 LEARNER]: Refusing update from remote peer ebcfe9bfa65f40c78193bd52e01d83a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:54:47.265024 23086 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 1 FOLLOWER]: Refusing update from remote peer ebcfe9bfa65f40c78193bd52e01d83a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:54:47.265128 23715 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:54:47.265406 23131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:36243: connect: Connection refused (error 111)
I20260806 07:54:47.266438 23715 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:54:47.267467 23131 consensus_peers.cc:597] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 -> Peer 17346d8291dc4d74a9f90f7d0023248b (127.20.202.254:36243): Couldn't send request to peer 17346d8291dc4d74a9f90f7d0023248b. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:36243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:54:47.269353 23715 raft_consensus.cc:2955] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.270077 23086 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.273849 23711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.274127 23711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:47.274521 23711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.274750 23711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:47.275113 23711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.275336 23711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:47.274943 23529 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.281847 23719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.282160 23719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3979fcaad53144bd856a9b9a8bd86b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:47.282661 23720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.282982 23720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31cac06e11ed4fb7b755795a9217252b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:54:47.454257 23710 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:47.454584 23710 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:47.454655 23710 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:47.463307 23710 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:54:47.463395 23710 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:47.463482 23710 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:54:47.463577 23710 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:54:47.473529 23710 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=33311
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:47.475247 23710 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:47.477159 23710 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:47.486193 23725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:47.486219 23726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:47.488700 23728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:47.489087 23710 server_base.cc:1061] running on GCE node
I20260806 07:54:47.489962 23710 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:47.490149 23710 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:47.490243 23710 hybrid_clock.cc:648] HybridClock initialized: now 1786002887490205 us; error 0 us; skew 500 ppm
I20260806 07:54:47.490665 23710 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:47.493405 23710 webserver.cc:533] Webserver started at http://127.20.202.254:33311/ using document root <none> and password file <none>
I20260806 07:54:47.494233 23710 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:47.494400 23710 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:47.500061 23710 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:54:47.503327 23733 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:47.505077 23710 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:54:47.505259 23710 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "17346d8291dc4d74a9f90f7d0023248b"
format_stamp: "Formatted at 2026-08-06 07:54:43 on dist-test-slave-g350"
I20260806 07:54:47.506088 23710 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:47.521279 23710 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:47.522209 23710 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:47.522513 23710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:47.543943 23710 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:36243
I20260806 07:54:47.543982 23784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:36243 every 8 connection(s)
I20260806 07:54:47.545780 23710 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:54:47.553040 23785 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:54:47.555864 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23710
I20260806 07:54:47.556206 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23493
I20260806 07:54:47.556663 23785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Bootstrap starting.
I20260806 07:54:47.565205 23690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:47.577100 23785 log.cc:826] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:54:47.586594 23785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:54:47.587217 23785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: Bootstrap complete.
I20260806 07:54:47.594492 23785 raft_consensus.cc:359] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } }
I20260806 07:54:47.595227 23785 raft_consensus.cc:740] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17346d8291dc4d74a9f90f7d0023248b, State: Initialized, Role: FOLLOWER
I20260806 07:54:47.596062 23785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } }
I20260806 07:54:47.596988 23791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: true } } }
I20260806 07:54:47.597307 23791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:47.598253 23785 sys_catalog.cc:565] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:54:47.609046 23802 catalog_manager.cc:1295] Loaded cluster ID: fd73481fd533440d94dc57feb090c63f
I20260806 07:54:47.609236 23802 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: loading cluster ID for follower catalog manager: success
I20260806 07:54:47.613615 23802 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: acquiring CA information for follower catalog manager: success
I20260806 07:54:47.616730 23802 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:54:47.657972 23760 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 31cac06e11ed4fb7b755795a9217252b (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.672593 23791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcfe9bfa65f40c78193bd52e01d83a2. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.672854 23791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:47.673146 23809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:47.673434 23809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17346d8291dc4d74a9f90f7d0023248b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:54:47.680804 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23493
I20260806 07:54:47.707353 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42177
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--webserver_port=35651
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865 with env {}
W20260806 07:54:48.129315 23811 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:48.129660 23811 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:48.129815 23811 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:48.138327 23811 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:48.138559 23811 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:48.148957 23811 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42177
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35651
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:48.150779 23811 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:48.152668 23811 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:48.162918 23819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:48.162961 23816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:48.162983 23817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:48.163911 23811 server_base.cc:1061] running on GCE node
I20260806 07:54:48.164467 23811 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:48.164660 23811 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:48.164760 23811 hybrid_clock.cc:648] HybridClock initialized: now 1786002888164717 us; error 0 us; skew 500 ppm
I20260806 07:54:48.165056 23811 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:48.167384 23811 webserver.cc:533] Webserver started at http://127.20.202.193:35651/ using document root <none> and password file <none>
I20260806 07:54:48.168344 23811 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:48.168464 23811 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:48.174357 23811 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:54:48.177114 23824 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:48.178694 23811 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:48.178864 23811 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:48.179672 23811 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:48.194298 23715 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "17346d8291dc4d74a9f90f7d0023248b" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36243 } } peers { permanent_uuid: "3979fcaad53144bd856a9b9a8bd86b41" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 46797 } } peers { permanent_uuid: "ebcfe9bfa65f40c78193bd52e01d83a2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 37357 } } peers { permanent_uuid: "31cac06e11ed4fb7b755795a9217252b" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 44865 } attrs { promote: false } } }
I20260806 07:54:48.194574 23715 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:54:48.195851 23811 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:48.196635 23811 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:48.196884 23811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:48.197844 23811 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:48.199757 23811 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:48.199872 23811 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:48.199980 23811 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:48.200024 23811 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:48.237545 23811 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42177
I20260806 07:54:48.237577 23940 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42177 every 8 connection(s)
I20260806 07:54:48.239171 23811 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:48.245723 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23811
I20260806 07:54:48.282138 23943 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36243
I20260806 07:54:48.282307 23942 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37357
I20260806 07:54:48.282577 23942 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:48.282574 23943 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:48.283752 23943 heartbeater.cc:507] Master 127.20.202.254:36243 requested a full tablet report, sending...
I20260806 07:54:48.284435 23942 heartbeater.cc:507] Master 127.20.202.252:37357 requested a full tablet report, sending...
I20260806 07:54:48.285367 23941 heartbeater.cc:344] Connected to a master server at 127.20.202.251:44865
I20260806 07:54:48.285521 23941 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:48.286019 23941 heartbeater.cc:507] Master 127.20.202.251:44865 requested a full tablet report, sending...
I20260806 07:54:48.286885 23750 ts_manager.cc:194] Registered new tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:48.287449 23519 ts_manager.cc:194] Re-registered known tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:48.288136 23147 ts_manager.cc:194] Re-registered known tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:48.289525 23147 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:37787
I20260806 07:54:48.293753 23944 heartbeater.cc:344] Connected to a master server at 127.20.202.253:46797
I20260806 07:54:48.293893 23944 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:48.294258 23944 heartbeater.cc:507] Master 127.20.202.253:46797 requested a full tablet report, sending...
I20260806 07:54:48.295317 23077 ts_manager.cc:194] Re-registered known tserver with Master: a019498148cf4368a068eb9ca9537c16 (127.20.202.193:42177)
I20260806 07:54:48.315541 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23811
I20260806 07:54:48.335601 23936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:48.424207 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23811
I20260806 07:54:48.446379 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42177
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--webserver_port=35651
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--fault_before_start=1.0 with env {}
W20260806 07:54:48.807986 23958 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:48.808336 23958 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:48.808483 23958 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:54:48.808570 23958 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:48.817416 23958 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:48.817684 23958 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:48.828128 23958 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42177
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35651
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:48.829931 23958 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:48.831866 23958 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:48.842257 23964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:48.842294 23963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:48.842833 23966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:48.843722 23958 server_base.cc:1061] running on GCE node
I20260806 07:54:48.844367 23958 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:48.844537 23958 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:48.844661 23958 hybrid_clock.cc:648] HybridClock initialized: now 1786002888844604 us; error 0 us; skew 500 ppm
I20260806 07:54:48.844975 23958 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:48.847366 23958 webserver.cc:533] Webserver started at http://127.20.202.193:35651/ using document root <none> and password file <none>
I20260806 07:54:48.848140 23958 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:48.848244 23958 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:48.853729 23958 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:54:48.856592 23971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:48.858173 23958 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:48.858307 23958 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:48.859062 23958 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:48.879652 23958 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:48.880527 23958 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:48.880767 23958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:48.881704 23958 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:48.883338 23958 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:48.883428 23958 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:48.883503 23958 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:48.883565 23958 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:54:48.883865 23958 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:54:48.884024 23958 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:54:48.884086 23958 fault_injection.cc:59] injecting fault for kudu (pid 23958): FLAGS_fault_before_start (process will exit)
I20260806 07:54:48.908551 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:42177
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357
--webserver_port=35651
--webserver_interface=127.20.202.193
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865 with env {}
W20260806 07:54:49.258982 23979 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:54:49.259326 23979 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:54:49.259487 23979 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:54:49.268525 23979 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:54:49.268750 23979 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:54:49.279142 23979 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:42177
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35651
--tserver_master_addrs=127.20.202.254:36243,127.20.202.253:46797,127.20.202.252:37357,127.20.202.251:44865
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:54:49.280855 23979 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:54:49.282873 23979 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:54:49.292742 23985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:49.292781 23984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:54:49.293687 23988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:54:49.294183 23979 server_base.cc:1061] running on GCE node
I20260806 07:54:49.294741 23979 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:54:49.294875 23979 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:54:49.294939 23979 hybrid_clock.cc:648] HybridClock initialized: now 1786002889294922 us; error 0 us; skew 500 ppm
I20260806 07:54:49.295259 23979 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:54:49.297375 23979 webserver.cc:533] Webserver started at http://127.20.202.193:35651/ using document root <none> and password file <none>
I20260806 07:54:49.298264 23979 fs_manager.cc:362] Metadata directory not provided
I20260806 07:54:49.298384 23979 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:54:49.304059 23979 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 07:54:49.306915 23993 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:49.308350 23979 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:54:49.308531 23979 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a019498148cf4368a068eb9ca9537c16"
format_stamp: "Formatted at 2026-08-06 07:54:45 on dist-test-slave-g350"
I20260806 07:54:49.309264 23979 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:54:49.332274 23979 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:54:49.333215 23979 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:54:49.333534 23979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:54:49.334522 23979 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:54:49.336412 23979 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:54:49.336512 23979 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:49.336678 23979 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:54:49.336751 23979 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:54:49.372534 23979 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42177
I20260806 07:54:49.372623 24105 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42177 every 8 connection(s)
I20260806 07:54:49.374228 23979 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:54:49.380018 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 23979
I20260806 07:54:49.380368 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23979
I20260806 07:54:49.406917 24106 heartbeater.cc:344] Connected to a master server at 127.20.202.251:44865
I20260806 07:54:49.407378 24106 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:49.407827 24101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:49.496112 24109 heartbeater.cc:344] Connected to a master server at 127.20.202.253:46797
I20260806 07:54:49.496286 24108 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36243
I20260806 07:54:49.496166 24107 heartbeater.cc:344] Connected to a master server at 127.20.202.252:37357
I20260806 07:54:49.496620 24109 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:49.496896 24108 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:49.497328 24107 heartbeater.cc:461] Registering TS with master...
I20260806 07:54:49.497973 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23979
I20260806 07:54:49.516889 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23710
I20260806 07:54:49.523026 23780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:49.629133 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23710
W20260806 07:54:49.651615 23131 connection.cc:573] client connection to 127.20.202.254:36243 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:49.651888 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23046
I20260806 07:54:49.651896 23114 master_runner.cc:217] Error getting master consensus for 127.20.202.254:36243: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:54:49.652119 23114 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:54:49.653132 23107 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:54:49.727960 23131 consensus_peers.cc:597] T 00000000000000000000000000000000 P ebcfe9bfa65f40c78193bd52e01d83a2 -> Peer 17346d8291dc4d74a9f90f7d0023248b (127.20.202.254:36243): Couldn't send request to peer 17346d8291dc4d74a9f90f7d0023248b. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:36243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:54:49.762343 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23046
I20260806 07:54:49.784626 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23114
I20260806 07:54:49.785849 23177 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:49.897508 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23114
I20260806 07:54:49.916294 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 23380
I20260806 07:54:49.917529 23549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:54:50.031471 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 23380
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestAddMaster/3 (23470 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/4
2026-08-06T07:54:50Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:54:50Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:54:58: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:55:06.472371 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45885
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659 with env {}
W20260806 07:55:06.823282 24412 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:06.823637 24412 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:06.823751 24412 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:06.831959 24412 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:06.832050 24412 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:06.832110 24412 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:06.832151 24412 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:06.842267 24412 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:06.843845 24412 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:06.845723 24412 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:06.855510 24418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:06.855583 24420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:06.856076 24417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:06.856261 24412 server_base.cc:1061] running on GCE node
I20260806 07:55:06.857131 24412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:06.858686 24412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:06.859964 24412 hybrid_clock.cc:648] HybridClock initialized: now 1786002906859895 us; error 55 us; skew 500 ppm
I20260806 07:55:06.860344 24412 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:06.862654 24412 webserver.cc:533] Webserver started at http://127.20.202.254:40355/ using document root <none> and password file <none>
I20260806 07:55:06.863449 24412 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:06.863554 24412 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:06.863976 24412 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:06.866475 24412 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "d2e5613bb63647ed814bb72bf8afda52"
format_stamp: "Formatted at 2026-08-06 07:55:06 on dist-test-slave-g350"
I20260806 07:55:06.867295 24412 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "d2e5613bb63647ed814bb72bf8afda52"
format_stamp: "Formatted at 2026-08-06 07:55:06 on dist-test-slave-g350"
I20260806 07:55:06.872764 24412 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:55:06.876761 24426 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:06.878255 24412 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:06.878450 24412 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "d2e5613bb63647ed814bb72bf8afda52"
format_stamp: "Formatted at 2026-08-06 07:55:06 on dist-test-slave-g350"
I20260806 07:55:06.878671 24412 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:06.922375 24412 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:06.923478 24412 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:06.923784 24412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:06.944825 24412 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45885
I20260806 07:55:06.944847 24478 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45885 every 8 connection(s)
I20260806 07:55:06.946504 24412 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:06.950819 24479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:06.956379 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24412
I20260806 07:55:06.956611 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:55:06.956844 24479 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:06.958039 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:32985
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659 with env {}
I20260806 07:55:06.974571 24479 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:06.975896 24429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:32985: connect: Connection refused (error 111)
W20260806 07:55:06.977851 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:32985: Network error: Client connection negotiation failed: client connection to 127.20.202.253:32985: connect: Connection refused (error 111)
I20260806 07:55:07.027312 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } attempt: 1
W20260806 07:55:07.029695 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:32985: Network error: Client connection negotiation failed: client connection to 127.20.202.253:32985: connect: Connection refused (error 111)
I20260806 07:55:07.098037 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } attempt: 2
W20260806 07:55:07.100253 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:32985: Network error: Client connection negotiation failed: client connection to 127.20.202.253:32985: connect: Connection refused (error 111)
I20260806 07:55:07.191601 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } attempt: 3
W20260806 07:55:07.193946 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:32985: Network error: Client connection negotiation failed: client connection to 127.20.202.253:32985: connect: Connection refused (error 111)
W20260806 07:55:07.326819 24483 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:07.327160 24483 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:07.327237 24483 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:07.335964 24483 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:07.336073 24483 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:07.336136 24483 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:07.336190 24483 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:07.337234 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } attempt: 4
W20260806 07:55:07.339633 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:32985: Network error: Client connection negotiation failed: client connection to 127.20.202.253:32985: connect: Connection refused (error 111)
I20260806 07:55:07.347386 24483 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:32985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:07.349189 24483 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:07.351234 24483 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:07.360327 24490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:07.360412 24489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:07.361045 24492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:07.362528 24483 server_base.cc:1061] running on GCE node
I20260806 07:55:07.363354 24483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:07.364868 24483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:07.366138 24483 hybrid_clock.cc:648] HybridClock initialized: now 1786002907366100 us; error 61 us; skew 500 ppm
I20260806 07:55:07.366549 24483 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:07.368836 24483 webserver.cc:533] Webserver started at http://127.20.202.253:35531/ using document root <none> and password file <none>
I20260806 07:55:07.369660 24483 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:07.369812 24483 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:07.370187 24483 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:07.372539 24483 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "379c0a9d2a68459da70122d69c540093"
format_stamp: "Formatted at 2026-08-06 07:55:07 on dist-test-slave-g350"
I20260806 07:55:07.373360 24483 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "379c0a9d2a68459da70122d69c540093"
format_stamp: "Formatted at 2026-08-06 07:55:07 on dist-test-slave-g350"
I20260806 07:55:07.378875 24483 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 07:55:07.382426 24498 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:07.383919 24483 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:07.384128 24483 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "379c0a9d2a68459da70122d69c540093"
format_stamp: "Formatted at 2026-08-06 07:55:07 on dist-test-slave-g350"
I20260806 07:55:07.384369 24483 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:07.418906 24483 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:07.419881 24483 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:07.420192 24483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:07.441255 24483 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:32985
I20260806 07:55:07.441274 24549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:32985 every 8 connection(s)
I20260806 07:55:07.444520 24483 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:55:07.449291 24550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:07.454918 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24483
I20260806 07:55:07.455171 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:55:07.456221 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38659
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659 with env {}
I20260806 07:55:07.456511 24550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:07.472398 24550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:07.481793 24550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:07.482872 24499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
W20260806 07:55:07.484714 24550 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
I20260806 07:55:07.534108 24550 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 1
W20260806 07:55:07.536751 24550 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
I20260806 07:55:07.605049 24550 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 2
W20260806 07:55:07.607462 24550 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
I20260806 07:55:07.638902 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } attempt: 5
I20260806 07:55:07.646252 24479 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:07.648144 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
I20260806 07:55:07.698746 24550 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 3
I20260806 07:55:07.699358 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 1
W20260806 07:55:07.701380 24550 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
W20260806 07:55:07.701478 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
I20260806 07:55:07.769727 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 2
W20260806 07:55:07.772152 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
W20260806 07:55:07.820389 24553 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:07.820748 24553 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:07.820842 24553 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:07.830433 24553 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:07.830543 24553 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:07.830613 24553 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:07.830664 24553 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:07.842375 24553 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:07.844185 24553 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:07.844621 24550 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 4
I20260806 07:55:07.847085 24553 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:07.847338 24550 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
W20260806 07:55:07.857170 24560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:07.857234 24561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:07.857568 24563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:07.858752 24553 server_base.cc:1061] running on GCE node
I20260806 07:55:07.859750 24553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:07.861500 24553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:07.862818 24553 hybrid_clock.cc:648] HybridClock initialized: now 1786002907862745 us; error 56 us; skew 500 ppm
I20260806 07:55:07.863225 24553 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:07.865401 24553 webserver.cc:533] Webserver started at http://127.20.202.252:36909/ using document root <none> and password file <none>
I20260806 07:55:07.866240 24553 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:07.866389 24553 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:07.866784 24553 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:07.869182 24553 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "9c418e37ea2c448188845d5d10e98ad2"
format_stamp: "Formatted at 2026-08-06 07:55:07 on dist-test-slave-g350"
I20260806 07:55:07.870077 24553 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "9c418e37ea2c448188845d5d10e98ad2"
format_stamp: "Formatted at 2026-08-06 07:55:07 on dist-test-slave-g350"
I20260806 07:55:07.875567 24553 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 07:55:07.878401 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 3
I20260806 07:55:07.879367 24569 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260806 07:55:07.880581 24479 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:38659: Network error: Client connection negotiation failed: client connection to 127.20.202.252:38659: connect: Connection refused (error 111)
I20260806 07:55:07.881091 24553 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:55:07.881289 24553 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "9c418e37ea2c448188845d5d10e98ad2"
format_stamp: "Formatted at 2026-08-06 07:55:07 on dist-test-slave-g350"
I20260806 07:55:07.881513 24553 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:07.908104 24553 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:07.909119 24553 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:07.909438 24553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:07.931306 24553 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:38659
I20260806 07:55:07.931327 24620 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:38659 every 8 connection(s)
I20260806 07:55:07.933012 24553 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:55:07.937704 24621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:07.942700 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24553
I20260806 07:55:07.942909 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:55:07.946393 24621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:07.969027 24621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:07.975371 24621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:07.985150 24621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: Bootstrap starting.
I20260806 07:55:07.988729 24621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:07.990063 24621 log.cc:826] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:07.992616 24621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: No bootstrap required, opened a new log
I20260806 07:55:07.998015 24621 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:07.998286 24621 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:07.998426 24621 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c418e37ea2c448188845d5d10e98ad2, State: Initialized, Role: FOLLOWER
I20260806 07:55:07.999047 24621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:07.999989 24628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.000286 24628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.001345 24621 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:08.012179 24639 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:08.012312 24639 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:08.008347 24553 master_runner.cc:187] Error getting master registration for 127.20.202.254:45885: OK, instance_id { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" instance_seqno: 1786002906939158 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:08.014134 24553 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:08.057819 24479 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 4
I20260806 07:55:08.068800 24479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Bootstrap starting.
I20260806 07:55:08.072355 24479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:08.073637 24479 log.cc:826] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:08.077412 24479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: No bootstrap required, opened a new log
I20260806 07:55:08.083184 24479 raft_consensus.cc:359] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.083458 24479 raft_consensus.cc:385] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:08.083700 24479 raft_consensus.cc:740] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e5613bb63647ed814bb72bf8afda52, State: Initialized, Role: FOLLOWER
I20260806 07:55:08.084498 24479 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.085680 24642 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.086027 24642 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.086951 24479 sys_catalog.cc:565] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:08.100296 24653 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:08.100428 24653 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:08.099781 24412 master_runner.cc:187] Error getting master registration for 127.20.202.253:32985: OK, instance_id { permanent_uuid: "379c0a9d2a68459da70122d69c540093" instance_seqno: 1786002907435310 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:08.105357 24412 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:08.146584 24550 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } attempt: 5
I20260806 07:55:08.160771 24550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: Bootstrap starting.
W20260806 07:55:08.161768 24575 tablet.cc:2406] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:08.165087 24550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:08.166567 24550 log.cc:826] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:08.172981 24550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: No bootstrap required, opened a new log
W20260806 07:55:08.177053 24432 tablet.cc:2406] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:08.180945 24550 raft_consensus.cc:359] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.181299 24550 raft_consensus.cc:385] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:08.181423 24550 raft_consensus.cc:740] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379c0a9d2a68459da70122d69c540093, State: Initialized, Role: FOLLOWER
I20260806 07:55:08.182226 24550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.183276 24656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.183784 24656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.184587 24550 sys_catalog.cc:565] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:08.206981 24483 master_runner.cc:264] No leader master found from master 379c0a9d2a68459da70122d69c540093
I20260806 07:55:08.207136 24483 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:55:08.212445 24668 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:08.212697 24668 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:08.225930 24642 raft_consensus.cc:493] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:55:08.226253 24642 raft_consensus.cc:515] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.228152 24642 leader_election.cc:290] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 379c0a9d2a68459da70122d69c540093 (127.20.202.253:32985), 9c418e37ea2c448188845d5d10e98ad2 (127.20.202.252:38659)
I20260806 07:55:08.229996 24525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2e5613bb63647ed814bb72bf8afda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379c0a9d2a68459da70122d69c540093" is_pre_election: true
I20260806 07:55:08.230624 24525 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e5613bb63647ed814bb72bf8afda52 in term 0.
I20260806 07:55:08.231374 24596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2e5613bb63647ed814bb72bf8afda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c418e37ea2c448188845d5d10e98ad2" is_pre_election: true
I20260806 07:55:08.231844 24596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e5613bb63647ed814bb72bf8afda52 in term 0.
I20260806 07:55:08.232745 24427 leader_election.cc:304] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c418e37ea2c448188845d5d10e98ad2, d2e5613bb63647ed814bb72bf8afda52; no voters:
I20260806 07:55:08.233855 24642 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:55:08.234010 24642 raft_consensus.cc:493] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:55:08.234119 24642 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:08.237695 24642 raft_consensus.cc:515] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.239604 24596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2e5613bb63647ed814bb72bf8afda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c418e37ea2c448188845d5d10e98ad2"
I20260806 07:55:08.239823 24596 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:08.243532 24596 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e5613bb63647ed814bb72bf8afda52 in term 1.
I20260806 07:55:08.244133 24427 leader_election.cc:304] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c418e37ea2c448188845d5d10e98ad2, d2e5613bb63647ed814bb72bf8afda52; no voters:
I20260806 07:55:08.244771 24525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2e5613bb63647ed814bb72bf8afda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379c0a9d2a68459da70122d69c540093"
I20260806 07:55:08.244985 24525 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:08.248611 24525 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e5613bb63647ed814bb72bf8afda52 in term 1.
I20260806 07:55:08.253520 24642 leader_election.cc:290] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [CANDIDATE]: Term 1 election: Requested vote from peers 379c0a9d2a68459da70122d69c540093 (127.20.202.253:32985), 9c418e37ea2c448188845d5d10e98ad2 (127.20.202.252:38659)
I20260806 07:55:08.254120 24642 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:55:08.254688 24642 raft_consensus.cc:697] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 LEADER]: Becoming Leader. State: Replica: d2e5613bb63647ed814bb72bf8afda52, State: Running, Role: LEADER
I20260806 07:55:08.256083 24642 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } }
I20260806 07:55:08.267244 24670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e5613bb63647ed814bb72bf8afda52. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.267721 24670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:08.279187 24672 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:55:08.281785 24672 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:55:08.298223 24596 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Refusing update from remote peer d2e5613bb63647ed814bb72bf8afda52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:08.303090 24670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:08.311400 24525 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Refusing update from remote peer d2e5613bb63647ed814bb72bf8afda52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:08.312500 24670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:08.349885 24656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e5613bb63647ed814bb72bf8afda52. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.350235 24656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.356256 24673 mvcc.cc:204] Tried to move back new op lower bound from 7315467912376123392 to 7315467912230068224. Current Snapshot: MvccSnapshot[applied={T|T < 7315467912376123392}]
I20260806 07:55:08.357813 24675 mvcc.cc:204] Tried to move back new op lower bound from 7315467912376123392 to 7315467912230068224. Current Snapshot: MvccSnapshot[applied={T|T < 7315467912376123392}]
I20260806 07:55:08.374126 24628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e5613bb63647ed814bb72bf8afda52. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.374449 24628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.380460 24628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.380710 24628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.390086 24656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.390332 24656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:08.394042 24642 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.394349 24642 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:08.394667 24642 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } }
I20260806 07:55:08.394888 24642 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:08.395771 24672 catalog_manager.cc:1383] Generated new cluster ID: ad7ac619c6fd4266ab12a473aef8ffeb
I20260806 07:55:08.395877 24672 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:55:08.397702 24674 mvcc.cc:204] Tried to move back new op lower bound from 7315467912376123392 to 7315467912230068224. Current Snapshot: MvccSnapshot[applied={T|T < 7315467912376123392 or (T in {7315467912376123392})}]
I20260806 07:55:08.435853 24672 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:55:08.437557 24672 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:55:08.466168 24672 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Generated new TSK 0
I20260806 07:55:08.467708 24672 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:55:08.507402 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:55:08.928345 24681 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:08.928637 24681 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:08.928771 24681 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:08.937096 24681 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:08.937288 24681 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:08.947817 24681 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:08.949815 24681 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:08.951855 24681 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:08.963392 24686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:08.965583 24687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:08.966363 24689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:08.967454 24681 server_base.cc:1061] running on GCE node
I20260806 07:55:08.968158 24681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:08.969261 24681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:08.970521 24681 hybrid_clock.cc:648] HybridClock initialized: now 1786002908970442 us; error 70 us; skew 500 ppm
I20260806 07:55:08.970875 24681 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:08.973204 24681 webserver.cc:533] Webserver started at http://127.20.202.193:35715/ using document root <none> and password file <none>
I20260806 07:55:08.974071 24681 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:08.974193 24681 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:08.974582 24681 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:08.977092 24681 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:08.977945 24681 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:08.983443 24681 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 07:55:08.987094 24695 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:08.988672 24681 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:08.988853 24681 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:08.989135 24681 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:09.008105 24681 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:09.008915 24681 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:09.009261 24681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:09.010281 24681 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:09.011969 24681 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:09.012094 24681 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:09.012233 24681 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:09.012323 24681 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:09.014184 24639 catalog_manager.cc:1295] Loaded cluster ID: ad7ac619c6fd4266ab12a473aef8ffeb
I20260806 07:55:09.014264 24639 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: loading cluster ID for follower catalog manager: success
I20260806 07:55:09.016754 24639 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: acquiring CA information for follower catalog manager: success
I20260806 07:55:09.019366 24639 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:09.047721 24681 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40003
I20260806 07:55:09.047811 24807 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40003 every 8 connection(s)
I20260806 07:55:09.049253 24681 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:09.050508 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24681
I20260806 07:55:09.050706 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:55:09.073496 24809 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45885
I20260806 07:55:09.073856 24808 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38659
I20260806 07:55:09.074087 24808 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:09.073557 24810 heartbeater.cc:344] Connected to a master server at 127.20.202.253:32985
I20260806 07:55:09.074102 24809 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:09.074329 24810 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:09.075421 24808 heartbeater.cc:507] Master 127.20.202.252:38659 requested a full tablet report, sending...
I20260806 07:55:09.075497 24809 heartbeater.cc:507] Master 127.20.202.254:45885 requested a full tablet report, sending...
I20260806 07:55:09.075870 24810 heartbeater.cc:507] Master 127.20.202.253:32985 requested a full tablet report, sending...
I20260806 07:55:09.078156 24444 ts_manager.cc:194] Registered new tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:09.078156 24586 ts_manager.cc:194] Registered new tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:09.078680 24515 ts_manager.cc:194] Registered new tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:09.080170 24444 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:58439
I20260806 07:55:09.094533 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:09.097157 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:33655
--webserver_interface=127.20.202.251
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655 with env {}
I20260806 07:55:09.214637 24668 catalog_manager.cc:1295] Loaded cluster ID: ad7ac619c6fd4266ab12a473aef8ffeb
I20260806 07:55:09.214730 24668 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: loading cluster ID for follower catalog manager: success
I20260806 07:55:09.217253 24668 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: acquiring CA information for follower catalog manager: success
I20260806 07:55:09.218976 24668 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:55:09.450587 24818 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:09.450999 24818 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:09.451133 24818 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:09.460317 24818 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:09.460445 24818 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:09.460565 24818 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:09.460646 24818 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:09.470875 24818 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:33655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:09.472529 24818 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:09.474460 24818 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:09.482551 24824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:09.482499 24823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:09.483392 24826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:09.483711 24818 server_base.cc:1061] running on GCE node
I20260806 07:55:09.484412 24818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:09.485874 24818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:09.487063 24818 hybrid_clock.cc:648] HybridClock initialized: now 1786002909487023 us; error 33 us; skew 500 ppm
I20260806 07:55:09.487401 24818 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:09.489831 24818 webserver.cc:533] Webserver started at http://127.20.202.251:38421/ using document root <none> and password file <none>
I20260806 07:55:09.490597 24818 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:09.490737 24818 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:09.491129 24818 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:09.493404 24818 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "2949249b27094306bdd8524c7cf25288"
format_stamp: "Formatted at 2026-08-06 07:55:09 on dist-test-slave-g350"
I20260806 07:55:09.494369 24818 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "2949249b27094306bdd8524c7cf25288"
format_stamp: "Formatted at 2026-08-06 07:55:09 on dist-test-slave-g350"
I20260806 07:55:09.499596 24818 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 07:55:09.503666 24832 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:09.505376 24818 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 07:55:09.505560 24818 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "2949249b27094306bdd8524c7cf25288"
format_stamp: "Formatted at 2026-08-06 07:55:09 on dist-test-slave-g350"
I20260806 07:55:09.505874 24818 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:09.522487 24818 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:09.523259 24818 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:09.523485 24818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:09.543989 24818 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:33655
I20260806 07:55:09.544032 24884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:33655 every 8 connection(s)
I20260806 07:55:09.545607 24818 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:09.549639 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24818
I20260806 07:55:09.550412 24885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:09.558693 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:09.569280 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:09.576129 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:09.582391 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:09.592451 24885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Bootstrap starting.
I20260806 07:55:09.595733 24885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:09.596810 24885 log.cc:826] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:09.599277 24885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: No bootstrap required, opened a new log
I20260806 07:55:09.604224 24885 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } }
I20260806 07:55:09.604570 24885 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:09.604708 24885 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2949249b27094306bdd8524c7cf25288, State: Initialized, Role: FOLLOWER
I20260806 07:55:09.605350 24885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } }
I20260806 07:55:09.606290 24892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } } }
I20260806 07:55:09.606580 24892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:09.607229 24885 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:09.617043 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24681
W20260806 07:55:09.617079 24903 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:09.617190 24903 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:09.626544 24818 master_runner.cc:429] Detected that this master 2949249b27094306bdd8524c7cf25288 is joining an existing cluster
I20260806 07:55:09.626670 24818 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:55:09.627609 24803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:09.628126 24818 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:55:09.628475 24818 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:55:09.628696 24818 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: stopping tablet replica
I20260806 07:55:09.630700 24818 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:33655
I20260806 07:55:09.667089 24444 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:33655: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2949249b27094306bdd8524c7cf25288"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 33655
}
attrs {
promote: true
}
}
dest_uuid: "d2e5613bb63647ed814bb72bf8afda52"
cas_config_opid_index: -1
I20260806 07:55:09.668910 24444 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:09.672034 24524 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Refusing update from remote peer d2e5613bb63647ed814bb72bf8afda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:55:09.672637 24430 consensus_peers.cc:597] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 -> Peer 2949249b27094306bdd8524c7cf25288 (127.20.202.251:33655): Couldn't send request to peer 2949249b27094306bdd8524c7cf25288. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:09.672794 24670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:09.675518 24596 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Refusing update from remote peer d2e5613bb63647ed814bb72bf8afda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:55:09.676203 24670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:09.686084 24918 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2949249b27094306bdd8524c7cf25288 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.687237 24524 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2949249b27094306bdd8524c7cf25288 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.693630 24595 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2949249b27094306bdd8524c7cf25288 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.701382 24918 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:33655
I20260806 07:55:09.702255 24818 master.cc:562] Master@127.20.202.251:33655 shutting down...
I20260806 07:55:09.702454 24920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.702752 24920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:09.710670 24921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.711311 24670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.711565 24670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:09.711468 24921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:09.717365 24918 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.717326 24923 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.717614 24918 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:09.717648 24923 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:09.750608 24818 master.cc:584] Master@127.20.202.251:33655 shutdown complete.
I20260806 07:55:09.750758 24818 master_runner.cc:306] Clearing existing system tablet
I20260806 07:55:09.751166 24818 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:09.751309 24818 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:09.753715 24818 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:55:09.755273 24926 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:09.755682 24818 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:09.755959 24818 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 07:55:09.756067 24818 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "2949249b27094306bdd8524c7cf25288"
format_stamp: "Formatted at 2026-08-06 07:55:09 on dist-test-slave-g350"
I20260806 07:55:09.759557 24818 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:55:09.764292 24818 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:55:09.764467 24818 log.cc:1199] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:55:09.764948 24818 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Deleting consensus metadata
I20260806 07:55:09.765204 24818 master_runner.cc:316] Copying system tablet from 127.20.202.254:45885
I20260806 07:55:09.768093 24818 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.254:45885
I20260806 07:55:09.779961 24464 tablet_copy_service.cc:140] P d2e5613bb63647ed814bb72bf8afda52: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2949249b27094306bdd8524c7cf25288 ({username='slave'} at 127.0.0.1:50686)
I20260806 07:55:09.780256 24464 tablet_copy_service.cc:161] P d2e5613bb63647ed814bb72bf8afda52: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2949249b27094306bdd8524c7cf25288 at {username='slave'} at 127.0.0.1:50686: session id = 2949249b27094306bdd8524c7cf25288-00000000000000000000000000000000
I20260806 07:55:09.783248 24464 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:55:09.785382 24818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:55:09.791106 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24681
I20260806 07:55:09.792402 24818 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: tablet copy: Starting download of 0 data blocks...
I20260806 07:55:09.792706 24818 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: tablet copy: Starting download of 1 WAL segments...
I20260806 07:55:09.798043 24818 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:55:09.801348 24464 tablet_copy_service.cc:342] P d2e5613bb63647ed814bb72bf8afda52: Request end of tablet copy session 2949249b27094306bdd8524c7cf25288-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:50686
I20260806 07:55:09.801587 24464 tablet_copy_service.cc:434] P d2e5613bb63647ed814bb72bf8afda52: ending tablet copy session 2949249b27094306bdd8524c7cf25288-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2949249b27094306bdd8524c7cf25288
W20260806 07:55:09.809034 24818 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:55:09.812369 24818 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:09.814862 24933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:09.818305 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40003
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--webserver_port=35715
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655 with env {}
W20260806 07:55:09.825927 24934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:09.828361 24818 server_base.cc:1061] running on GCE node
W20260806 07:55:09.828531 24936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:09.829012 24818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:09.829764 24818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:09.830988 24818 hybrid_clock.cc:648] HybridClock initialized: now 1786002909830904 us; error 52 us; skew 500 ppm
I20260806 07:55:09.831235 24818 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:09.832917 24818 webserver.cc:533] Webserver started at http://127.20.202.251:44005/ using document root <none> and password file <none>
I20260806 07:55:09.833346 24818 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:09.833453 24818 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:09.836750 24818 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:55:09.838827 24943 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:09.839346 24818 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 07:55:09.839533 24818 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "2949249b27094306bdd8524c7cf25288"
format_stamp: "Formatted at 2026-08-06 07:55:09 on dist-test-slave-g350"
I20260806 07:55:09.839766 24818 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:09.854647 24818 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:09.855727 24818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:09.881934 24818 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:33655
I20260806 07:55:09.881989 24994 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:33655 every 8 connection(s)
I20260806 07:55:09.883186 24818 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:09.888164 24995 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:09.892102 24995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Bootstrap starting.
I20260806 07:55:09.908711 24995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:09.909225 24995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: Bootstrap complete.
I20260806 07:55:09.910351 24995 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:09.910629 24995 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2949249b27094306bdd8524c7cf25288, State: Initialized, Role: LEARNER
I20260806 07:55:09.910888 24995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:09.911803 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:09.912066 24995 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:09.912247 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:09.922056 25009 catalog_manager.cc:1295] Loaded cluster ID: ad7ac619c6fd4266ab12a473aef8ffeb
I20260806 07:55:09.922183 25009 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: loading cluster ID for follower catalog manager: success
I20260806 07:55:09.926040 25009 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: acquiring CA information for follower catalog manager: success
I20260806 07:55:09.928622 25009 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:55:10.223373 24939 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:10.223664 24939 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:10.223793 24939 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:10.232079 24939 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:10.232295 24939 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:10.242333 24939 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40003
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35715
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:10.244009 24939 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:10.245895 24939 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:10.256402 25014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:10.256691 25017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:10.256726 25015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:10.257969 24939 server_base.cc:1061] running on GCE node
I20260806 07:55:10.258605 24939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:10.259750 24939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:10.260977 24939 hybrid_clock.cc:648] HybridClock initialized: now 1786002910260925 us; error 85 us; skew 500 ppm
I20260806 07:55:10.261276 24939 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:10.263495 24939 webserver.cc:533] Webserver started at http://127.20.202.193:35715/ using document root <none> and password file <none>
I20260806 07:55:10.264227 24939 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:10.264331 24939 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:10.270743 24939 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:55:10.273758 25025 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:10.275494 24939 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:55:10.276053 24970 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:55:10.275702 24939 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:10.276546 24939 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:10.276841 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e5613bb63647ed814bb72bf8afda52. Latest consensus state: current_term: 1 leader_uuid: "d2e5613bb63647ed814bb72bf8afda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:10.277094 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:10.311266 24939 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:10.312125 24939 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:10.312381 24939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:10.313392 24939 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:10.315102 24939 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:10.315174 24939 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:10.315256 24939 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:10.315299 24939 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:10.351498 24939 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40003
I20260806 07:55:10.351593 25137 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40003 every 8 connection(s)
I20260806 07:55:10.353121 24939 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:10.362411 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 24939
I20260806 07:55:10.409844 25138 heartbeater.cc:344] Connected to a master server at 127.20.202.251:33655
I20260806 07:55:10.410372 25138 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:10.411558 25138 heartbeater.cc:507] Master 127.20.202.251:33655 requested a full tablet report, sending...
I20260806 07:55:10.414876 24960 ts_manager.cc:194] Registered new tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:10.415894 25141 heartbeater.cc:344] Connected to a master server at 127.20.202.253:32985
I20260806 07:55:10.416062 25139 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38659
I20260806 07:55:10.416087 25141 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:10.416255 25139 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:10.416320 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:10.416707 25141 heartbeater.cc:507] Master 127.20.202.253:32985 requested a full tablet report, sending...
I20260806 07:55:10.416955 25139 heartbeater.cc:507] Master 127.20.202.252:38659 requested a full tablet report, sending...
I20260806 07:55:10.418534 24586 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:10.418553 24514 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:10.419554 25140 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45885
I20260806 07:55:10.419763 25140 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:10.420212 25140 heartbeater.cc:507] Master 127.20.202.254:45885 requested a full tablet report, sending...
I20260806 07:55:10.421445 24444 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:10.422449 24444 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:53985
I20260806 07:55:10.425575 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:10.425823 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24412
I20260806 07:55:10.426810 24474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:10.541567 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24412
I20260806 07:55:10.564992 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45885
--webserver_interface=127.20.202.254
--webserver_port=40355
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655 with env {}
W20260806 07:55:10.937165 25153 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:10.937489 25153 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:10.937660 25153 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:10.946272 25153 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:10.946360 25153 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:10.946458 25153 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:10.946542 25153 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:10.956730 25153 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=40355
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:10.958482 25153 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:10.960352 25153 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:10.968820 25158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:10.968896 25159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:10.969764 25161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:10.970135 25153 server_base.cc:1061] running on GCE node
I20260806 07:55:10.970870 25153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:10.972247 25153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:10.973495 25153 hybrid_clock.cc:648] HybridClock initialized: now 1786002910973453 us; error 49 us; skew 500 ppm
I20260806 07:55:10.973883 25153 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:10.976045 25153 webserver.cc:533] Webserver started at http://127.20.202.254:40355/ using document root <none> and password file <none>
I20260806 07:55:10.976805 25153 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:10.976969 25153 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:10.982537 25153 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:55:10.985327 25167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:10.986784 25153 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:55:10.986976 25153 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "d2e5613bb63647ed814bb72bf8afda52"
format_stamp: "Formatted at 2026-08-06 07:55:06 on dist-test-slave-g350"
I20260806 07:55:10.987721 25153 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:11.024118 25153 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:11.025086 25153 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:11.025403 25153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:11.046885 25153 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45885
I20260806 07:55:11.046885 25218 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45885 every 8 connection(s)
I20260806 07:55:11.048593 25153 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:11.055857 25219 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:11.057097 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25153
I20260806 07:55:11.057408 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24939
I20260806 07:55:11.060513 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Bootstrap starting.
I20260806 07:55:11.067040 25133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:11.080277 25219 log.cc:826] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:11.091223 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:11.091809 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: Bootstrap complete.
I20260806 07:55:11.098863 25219 raft_consensus.cc:359] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:11.099669 25219 raft_consensus.cc:740] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e5613bb63647ed814bb72bf8afda52, State: Initialized, Role: FOLLOWER
I20260806 07:55:11.100423 25219 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:11.101338 25225 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:11.101686 25225 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:11.102555 25219 sys_catalog.cc:565] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:11.120532 25237 catalog_manager.cc:1295] Loaded cluster ID: ad7ac619c6fd4266ab12a473aef8ffeb
I20260806 07:55:11.120653 25237 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: loading cluster ID for follower catalog manager: success
I20260806 07:55:11.125002 25237 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: acquiring CA information for follower catalog manager: success
I20260806 07:55:11.127731 25237 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:11.168951 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24939
I20260806 07:55:11.177734 25153 master_runner.cc:264] No leader master found from master d2e5613bb63647ed814bb72bf8afda52
I20260806 07:55:11.177937 25153 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:11.189246 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40003
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--webserver_port=35715
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655 with env {}
W20260806 07:55:11.556433 25244 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:11.556757 25244 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:11.556833 25244 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:11.565294 25244 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:11.565464 25244 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:11.573000 25248 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2e5613bb63647ed814bb72bf8afda52)
I20260806 07:55:11.573259 25248 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:11.575047 25248 leader_election.cc:290] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2e5613bb63647ed814bb72bf8afda52 (127.20.202.254:45885), 379c0a9d2a68459da70122d69c540093 (127.20.202.253:32985)
I20260806 07:55:11.575876 24524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c418e37ea2c448188845d5d10e98ad2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "379c0a9d2a68459da70122d69c540093" is_pre_election: true
I20260806 07:55:11.577306 25244 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40003
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35715
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:11.579015 25244 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:11.581020 25244 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:55:11.582612 25194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c418e37ea2c448188845d5d10e98ad2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "d2e5613bb63647ed814bb72bf8afda52" is_pre_election: true
I20260806 07:55:11.583120 25194 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c418e37ea2c448188845d5d10e98ad2 in term 1.
I20260806 07:55:11.583724 24570 leader_election.cc:304] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c418e37ea2c448188845d5d10e98ad2, d2e5613bb63647ed814bb72bf8afda52; no voters: 379c0a9d2a68459da70122d69c540093
I20260806 07:55:11.584131 25248 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:55:11.584277 25248 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2e5613bb63647ed814bb72bf8afda52)
I20260806 07:55:11.584388 25248 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:55:11.587690 25248 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:11.588635 25248 leader_election.cc:290] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [CANDIDATE]: Term 2 election: Requested vote from peers d2e5613bb63647ed814bb72bf8afda52 (127.20.202.254:45885), 379c0a9d2a68459da70122d69c540093 (127.20.202.253:32985)
I20260806 07:55:11.589215 25194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c418e37ea2c448188845d5d10e98ad2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "d2e5613bb63647ed814bb72bf8afda52"
I20260806 07:55:11.589483 25194 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:55:11.589781 24524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c418e37ea2c448188845d5d10e98ad2" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "379c0a9d2a68459da70122d69c540093"
I20260806 07:55:11.593758 25194 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c418e37ea2c448188845d5d10e98ad2 in term 2.
I20260806 07:55:11.594286 24570 leader_election.cc:304] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c418e37ea2c448188845d5d10e98ad2, d2e5613bb63647ed814bb72bf8afda52; no voters: 379c0a9d2a68459da70122d69c540093
I20260806 07:55:11.594636 25248 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:55:11.595095 25248 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 2 LEADER]: Becoming Leader. State: Replica: 9c418e37ea2c448188845d5d10e98ad2, State: Running, Role: LEADER
I20260806 07:55:11.595712 25248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
W20260806 07:55:11.596617 25252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:11.597882 25255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:11.598176 25251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:11.598893 25254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c418e37ea2c448188845d5d10e98ad2. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:11.599203 25254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:11.599186 25244 server_base.cc:1061] running on GCE node
I20260806 07:55:11.599777 25244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:11.599987 25258 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:55:11.600832 25244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:11.602100 25244 hybrid_clock.cc:648] HybridClock initialized: now 1786002911602024 us; error 63 us; skew 500 ppm
I20260806 07:55:11.602430 25258 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:55:11.602452 25244 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:11.603327 25258 catalog_manager.cc:1295] Loaded cluster ID: ad7ac619c6fd4266ab12a473aef8ffeb
I20260806 07:55:11.603400 25258 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:55:11.604703 25258 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:55:11.604986 25244 webserver.cc:533] Webserver started at http://127.20.202.193:35715/ using document root <none> and password file <none>
I20260806 07:55:11.605901 25258 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: Loaded TSK: 0
I20260806 07:55:11.605995 25244 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:11.606114 25244 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:11.606899 25258 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:55:11.611783 25244 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 07:55:11.614691 25263 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:11.616151 25244 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:55:11.616333 25244 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:11.617055 25244 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:11.629855 25244 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:11.630690 25244 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:11.630983 25244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:11.631881 25244 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:11.633493 25244 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:11.633594 25244 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:11.633767 25244 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:11.633839 25244 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:11.650318 25316 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2e5613bb63647ed814bb72bf8afda52)
I20260806 07:55:11.650709 25316 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } }
I20260806 07:55:11.652501 25316 leader_election.cc:290] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2e5613bb63647ed814bb72bf8afda52 (127.20.202.254:45885), 9c418e37ea2c448188845d5d10e98ad2 (127.20.202.252:38659)
I20260806 07:55:11.654212 24595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379c0a9d2a68459da70122d69c540093" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "9c418e37ea2c448188845d5d10e98ad2" is_pre_election: true
I20260806 07:55:11.661235 25194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379c0a9d2a68459da70122d69c540093" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "d2e5613bb63647ed814bb72bf8afda52" is_pre_election: true
I20260806 07:55:11.661705 25194 raft_consensus.cc:2393] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 379c0a9d2a68459da70122d69c540093 in current term 2: Already voted for candidate 9c418e37ea2c448188845d5d10e98ad2 in this term.
I20260806 07:55:11.662504 24499 leader_election.cc:304] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379c0a9d2a68459da70122d69c540093; no voters: 9c418e37ea2c448188845d5d10e98ad2, d2e5613bb63647ed814bb72bf8afda52
I20260806 07:55:11.662891 25316 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:55:11.665923 25316 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:55:11.670269 25244 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40003
I20260806 07:55:11.670346 25378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40003 every 8 connection(s)
I20260806 07:55:11.671833 25244 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:11.681874 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25244
I20260806 07:55:11.706766 25382 heartbeater.cc:344] Connected to a master server at 127.20.202.253:32985
I20260806 07:55:11.707229 25382 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:11.708407 25382 heartbeater.cc:507] Master 127.20.202.253:32985 requested a full tablet report, sending...
I20260806 07:55:11.708844 25379 heartbeater.cc:344] Connected to a master server at 127.20.202.251:33655
I20260806 07:55:11.709014 25379 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:11.709568 25379 heartbeater.cc:507] Master 127.20.202.251:33655 requested a full tablet report, sending...
I20260806 07:55:11.710656 24514 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:11.711117 25381 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45885
I20260806 07:55:11.711143 24959 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:11.711261 25381 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:11.711762 25381 heartbeater.cc:507] Master 127.20.202.254:45885 requested a full tablet report, sending...
I20260806 07:55:11.713048 25380 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38659
I20260806 07:55:11.713181 25380 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:11.713656 25380 heartbeater.cc:507] Master 127.20.202.252:38659 requested a full tablet report, sending...
I20260806 07:55:11.713840 25184 ts_manager.cc:194] Registered new tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:11.714622 24586 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:11.716410 24586 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:58861
I20260806 07:55:11.740496 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25244
I20260806 07:55:11.750660 25374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:11.842005 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25244
I20260806 07:55:11.861106 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40003
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--webserver_port=35715
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--fault_before_start=1.0 with env {}
I20260806 07:55:12.001279 25194 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 2 FOLLOWER]: Refusing update from remote peer 9c418e37ea2c448188845d5d10e98ad2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:55:12.002403 25254 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:12.006812 25396 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c418e37ea2c448188845d5d10e98ad2. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.007153 25396 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:12.009732 24524 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 2 FOLLOWER]: Refusing update from remote peer 9c418e37ea2c448188845d5d10e98ad2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:55:12.010613 25254 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:12.019515 24970 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 1 LEARNER]: Advancing to term 2
I20260806 07:55:12.024356 24970 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 2 LEARNER]: Refusing update from remote peer 9c418e37ea2c448188845d5d10e98ad2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20260806 07:55:12.019062 25248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.025223 25248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:12.025149 25254 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:12.030177 25396 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.030486 25396 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:12.032567 25316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c418e37ea2c448188845d5d10e98ad2. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.032823 25316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:12.032577 25405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.034003 25405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:12.034775 25408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.035141 25408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:12.035392 25403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c418e37ea2c448188845d5d10e98ad2. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: true } } }
I20260806 07:55:12.035733 25403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:12.036172 25254 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2: attempting to promote NON_VOTER 2949249b27094306bdd8524c7cf25288 to VOTER
I20260806 07:55:12.037365 25254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } }
I20260806 07:55:12.039629 24524 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 2 FOLLOWER]: Refusing update from remote peer 9c418e37ea2c448188845d5d10e98ad2: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:55:12.039794 24970 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 2 LEARNER]: Refusing update from remote peer 9c418e37ea2c448188845d5d10e98ad2: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:55:12.039801 25194 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 2 FOLLOWER]: Refusing update from remote peer 9c418e37ea2c448188845d5d10e98ad2: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:55:12.040445 25254 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:55:12.040736 25399 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:55:12.041270 25248 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:55:12.044333 25248 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2949249b27094306bdd8524c7cf25288 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.045181 24525 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2949249b27094306bdd8524c7cf25288 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.048293 25248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.048501 25248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:12.049264 25254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.049455 25254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:12.049649 25194 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2949249b27094306bdd8524c7cf25288 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.054100 25399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.054383 25399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:12.054615 25399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.054827 25399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:12.055473 25405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.056016 25396 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.056236 25396 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e5613bb63647ed814bb72bf8afda52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:12.056373 25405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379c0a9d2a68459da70122d69c540093 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:12.056171 24970 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, 2949249b27094306bdd8524c7cf25288 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.062326 25403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c418e37ea2c448188845d5d10e98ad2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e5613bb63647ed814bb72bf8afda52" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45885 } } peers { permanent_uuid: "379c0a9d2a68459da70122d69c540093" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 32985 } } peers { permanent_uuid: "9c418e37ea2c448188845d5d10e98ad2" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 38659 } } peers { permanent_uuid: "2949249b27094306bdd8524c7cf25288" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 33655 } attrs { promote: false } } }
I20260806 07:55:12.062552 25403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2949249b27094306bdd8524c7cf25288 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:55:12.249985 25395 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:12.250411 25395 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:12.250564 25395 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:55:12.250666 25395 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:12.260754 25395 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:12.261040 25395 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:12.271951 25395 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40003
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35715
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:12.273928 25395 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:12.275926 25395 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:12.286262 25415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:12.286301 25414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:12.287415 25417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:12.287808 25395 server_base.cc:1061] running on GCE node
I20260806 07:55:12.288488 25395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:12.289564 25395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:12.290874 25395 hybrid_clock.cc:648] HybridClock initialized: now 1786002912290791 us; error 62 us; skew 500 ppm
I20260806 07:55:12.291198 25395 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:12.293711 25395 webserver.cc:533] Webserver started at http://127.20.202.193:35715/ using document root <none> and password file <none>
I20260806 07:55:12.294479 25395 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:12.294634 25395 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:12.300263 25395 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:55:12.303192 25423 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:12.304792 25395 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:12.304998 25395 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:12.305821 25395 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:12.344013 25395 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:12.344905 25395 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:12.345161 25395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:12.346150 25395 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:12.347874 25395 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:12.347961 25395 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:12.348032 25395 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:12.348086 25395 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:55:12.348385 25395 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:55:12.348507 25395 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:55:12.348608 25395 fault_injection.cc:59] injecting fault for kudu (pid 25395): FLAGS_fault_before_start (process will exit)
I20260806 07:55:12.365522 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40003
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659
--webserver_port=35715
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655 with env {}
W20260806 07:55:12.720191 25431 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:12.720477 25431 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:12.720551 25431 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:12.728701 25431 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:12.728855 25431 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:12.738562 25431 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40003
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35715
--tserver_master_addrs=127.20.202.254:45885,127.20.202.253:32985,127.20.202.252:38659,127.20.202.251:33655
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:12.740149 25431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:12.742080 25431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:12.751622 25437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:12.752260 25436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:12.752534 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:12.753297 25431 server_base.cc:1061] running on GCE node
I20260806 07:55:12.753909 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:12.754845 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:12.756074 25431 hybrid_clock.cc:648] HybridClock initialized: now 1786002912756022 us; error 46 us; skew 500 ppm
I20260806 07:55:12.756409 25431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:12.758737 25431 webserver.cc:533] Webserver started at http://127.20.202.193:35715/ using document root <none> and password file <none>
I20260806 07:55:12.759538 25431 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:12.759684 25431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:12.765164 25431 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:55:12.768059 25445 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:12.769418 25431 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:55:12.769650 25431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "58a9dc5106234bd9a4c3d9884ee82e6e"
format_stamp: "Formatted at 2026-08-06 07:55:08 on dist-test-slave-g350"
I20260806 07:55:12.770463 25431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:12.791970 25431 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:12.792914 25431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:12.793279 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:12.794353 25431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:12.796118 25431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:12.796233 25431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:12.796391 25431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:12.796465 25431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:12.831979 25431 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40003
I20260806 07:55:12.832041 25558 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40003 every 8 connection(s)
I20260806 07:55:12.833638 25431 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:12.838196 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25431
I20260806 07:55:12.838567 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25431
I20260806 07:55:12.872828 25560 heartbeater.cc:344] Connected to a master server at 127.20.202.252:38659
I20260806 07:55:12.873229 25560 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:12.874359 25560 heartbeater.cc:507] Master 127.20.202.252:38659 requested a full tablet report, sending...
I20260806 07:55:12.876160 25559 heartbeater.cc:344] Connected to a master server at 127.20.202.251:33655
I20260806 07:55:12.876329 25559 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:12.876454 24586 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:12.876885 25559 heartbeater.cc:507] Master 127.20.202.251:33655 requested a full tablet report, sending...
I20260806 07:55:12.877609 24586 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:60207
I20260806 07:55:12.878154 24959 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:12.882570 25562 heartbeater.cc:344] Connected to a master server at 127.20.202.253:32985
I20260806 07:55:12.882737 25562 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:12.883167 25562 heartbeater.cc:507] Master 127.20.202.253:32985 requested a full tablet report, sending...
I20260806 07:55:12.884330 24514 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
I20260806 07:55:12.886595 25561 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45885
I20260806 07:55:12.886765 25561 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:12.886948 25554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:12.976141 25561 heartbeater.cc:507] Master 127.20.202.254:45885 requested a full tablet report, sending...
I20260806 07:55:12.977370 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25431
I20260806 07:55:12.977820 25184 ts_manager.cc:194] Re-registered known tserver with Master: 58a9dc5106234bd9a4c3d9884ee82e6e (127.20.202.193:40003)
W20260806 07:55:12.995725 25168 connection.cc:573] server connection from 127.20.202.193:49815 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:55:12.996156 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25153
I20260806 07:55:13.002449 25214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:13.112106 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25153
W20260806 07:55:13.133965 24570 connection.cc:573] client connection to 127.20.202.254:45885 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:55:13.134196 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24483
I20260806 07:55:13.134317 24553 master_runner.cc:217] Error getting master consensus for 127.20.202.254:45885: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:55:13.134666 24553 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:13.135499 24545 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:55:13.206091 24570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:45885: connect: Connection refused (error 111)
W20260806 07:55:13.210870 24570 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9c418e37ea2c448188845d5d10e98ad2 -> Peer d2e5613bb63647ed814bb72bf8afda52 (127.20.202.254:45885): Couldn't send request to peer d2e5613bb63647ed814bb72bf8afda52. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:45885: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:13.248824 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24483
I20260806 07:55:13.272190 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24553
I20260806 07:55:13.273473 24616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:13.385942 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24553
I20260806 07:55:13.408111 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 24818
I20260806 07:55:13.409376 24990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:13.522125 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 24818
Shutting down hive metastore.
2026-08-06T07:55:13Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/4 (23533 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/5
2026-08-06T07:55:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:55:14Z Disabled control of system clock
2026-08-06T07:55:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:55:14Z Disabled control of system clock
2026-08-06T07:55:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:55:14Z Disabled control of system clock
2026-08-06T07:55:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:55:14Z Disabled control of system clock
2026-08-06T07:55:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:55:14Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/security.properties
2026-08-06 07:55:21: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 07:55:30.155838 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35663
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135 with env {}
W20260806 07:55:30.502318 25871 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:30.502665 25871 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:30.502808 25871 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:30.511286 25871 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:30.511379 25871 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:30.511471 25871 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:30.511545 25871 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:30.522150 25871 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:30.523818 25871 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:30.525808 25871 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:30.534246 25877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:30.534320 25876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:30.534909 25879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:30.535356 25871 server_base.cc:1061] running on GCE node
I20260806 07:55:30.536146 25871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:30.537810 25871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:30.539088 25871 hybrid_clock.cc:648] HybridClock initialized: now 1786002930538945 us; error 116 us; skew 500 ppm
I20260806 07:55:30.539446 25871 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:30.541795 25871 webserver.cc:533] Webserver started at http://127.20.202.254:44649/ using document root <none> and password file <none>
I20260806 07:55:30.542555 25871 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:30.542656 25871 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:30.543069 25871 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:30.545449 25871 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "73415f06b3c34da8b3241cb15712dde8"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:30.546370 25871 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "73415f06b3c34da8b3241cb15712dde8"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:30.551658 25871 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:55:30.555027 25885 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:30.556346 25871 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:30.556533 25871 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "73415f06b3c34da8b3241cb15712dde8"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:30.556769 25871 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:30.578441 25871 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:30.579366 25871 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:30.579672 25871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:30.602339 25871 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:35663
I20260806 07:55:30.602347 25936 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:35663 every 8 connection(s)
I20260806 07:55:30.603924 25871 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:30.607008 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25871
I20260806 07:55:30.607223 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:55:30.608255 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34379
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135 with env {}
I20260806 07:55:30.608498 25937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:30.614346 25937 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:30.633777 25937 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:30.635097 25889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34379: connect: Connection refused (error 111)
W20260806 07:55:30.636936 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34379: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34379: connect: Connection refused (error 111)
I20260806 07:55:30.686261 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } attempt: 1
W20260806 07:55:30.688517 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34379: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34379: connect: Connection refused (error 111)
I20260806 07:55:30.756774 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } attempt: 2
W20260806 07:55:30.758973 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34379: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34379: connect: Connection refused (error 111)
I20260806 07:55:30.850256 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } attempt: 3
W20260806 07:55:30.852610 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34379: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34379: connect: Connection refused (error 111)
W20260806 07:55:30.962692 25941 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:30.963024 25941 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:30.963178 25941 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:30.971608 25941 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:30.971693 25941 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:30.971779 25941 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:30.971868 25941 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:30.982191 25941 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:30.983820 25941 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:30.985695 25941 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:30.993806 25947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:30.994040 25948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:30.994730 25950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:30.995230 25941 server_base.cc:1061] running on GCE node
I20260806 07:55:30.995867 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } attempt: 4
I20260806 07:55:30.995934 25941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260806 07:55:30.997895 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34379: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34379: connect: Connection refused (error 111)
I20260806 07:55:30.998077 25941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:30.999318 25941 hybrid_clock.cc:648] HybridClock initialized: now 1786002930999177 us; error 108 us; skew 500 ppm
I20260806 07:55:30.999675 25941 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:31.001880 25941 webserver.cc:533] Webserver started at http://127.20.202.253:46227/ using document root <none> and password file <none>
I20260806 07:55:31.002627 25941 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:31.002763 25941 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:31.003121 25941 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:31.005445 25941 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "d8c12f187e7f43ffacab7814ad6b2903"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:31.006316 25941 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "d8c12f187e7f43ffacab7814ad6b2903"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:31.011516 25941 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:55:31.014853 25956 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:31.016225 25941 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:31.016508 25941 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "d8c12f187e7f43ffacab7814ad6b2903"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:31.016736 25941 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:31.030164 25941 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:31.030989 25941 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:31.031296 25941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:31.051728 25941 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:34379
I20260806 07:55:31.051736 26008 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:34379 every 8 connection(s)
I20260806 07:55:31.053364 25941 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:55:31.057906 26009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:31.063182 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 25941
I20260806 07:55:31.063381 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:55:31.064817 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:39135
--webserver_interface=127.20.202.252
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135 with env {}
I20260806 07:55:31.064596 26009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:31.080222 26009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:31.089537 26009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:31.090832 25958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
W20260806 07:55:31.092558 26009 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
I20260806 07:55:31.141919 26009 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 1
W20260806 07:55:31.144239 26009 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
I20260806 07:55:31.212503 26009 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 2
W20260806 07:55:31.214748 26009 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
I20260806 07:55:31.297158 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } attempt: 5
I20260806 07:55:31.305742 25937 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:31.306000 26009 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 3
W20260806 07:55:31.308347 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
W20260806 07:55:31.308805 26009 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
I20260806 07:55:31.359622 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 1
W20260806 07:55:31.361924 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
W20260806 07:55:31.425021 26012 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:31.425336 26012 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:31.425407 26012 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:55:31.430178 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 2
W20260806 07:55:31.432255 25937 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
W20260806 07:55:31.434291 26012 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:31.434379 26012 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:31.434435 26012 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:31.434487 26012 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:31.444566 26012 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:39135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:31.446353 26012 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:31.448123 26012 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:55:31.452032 26009 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 4
W20260806 07:55:31.454130 26009 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:39135: Network error: Client connection negotiation failed: client connection to 127.20.202.252:39135: connect: Connection refused (error 111)
W20260806 07:55:31.457083 26020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:31.457103 26019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:31.457897 26022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:31.458518 26012 server_base.cc:1061] running on GCE node
I20260806 07:55:31.459203 26012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:31.460721 26012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:31.462030 26012 hybrid_clock.cc:648] HybridClock initialized: now 1786002931461914 us; error 81 us; skew 500 ppm
I20260806 07:55:31.462380 26012 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:31.464459 26012 webserver.cc:533] Webserver started at http://127.20.202.252:38683/ using document root <none> and password file <none>
I20260806 07:55:31.465193 26012 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:31.465312 26012 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:31.465740 26012 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:31.468122 26012 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "365ea10e275c44f6b99bbbd1dbd97bbc"
format_stamp: "Formatted at 2026-08-06 07:55:31 on dist-test-slave-g350"
I20260806 07:55:31.468889 26012 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "365ea10e275c44f6b99bbbd1dbd97bbc"
format_stamp: "Formatted at 2026-08-06 07:55:31 on dist-test-slave-g350"
I20260806 07:55:31.474136 26012 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:55:31.477586 26028 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:31.479173 26012 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:31.479377 26012 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "365ea10e275c44f6b99bbbd1dbd97bbc"
format_stamp: "Formatted at 2026-08-06 07:55:31 on dist-test-slave-g350"
I20260806 07:55:31.479598 26012 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:31.502254 26012 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:31.503145 26012 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:31.503435 26012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:31.524644 26012 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:39135
I20260806 07:55:31.524683 26079 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:39135 every 8 connection(s)
I20260806 07:55:31.526293 26012 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:55:31.530701 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26012
I20260806 07:55:31.530913 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:55:31.531095 26080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:31.538486 25937 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 3
I20260806 07:55:31.539623 26080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:31.554270 26080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:31.559119 25937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Bootstrap starting.
I20260806 07:55:31.561120 26080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:31.564785 25937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:31.566083 25937 log.cc:826] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:31.568749 25937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: No bootstrap required, opened a new log
I20260806 07:55:31.570819 26080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: Bootstrap starting.
I20260806 07:55:31.574174 26080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:31.574628 25937 raft_consensus.cc:359] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.574867 25937 raft_consensus.cc:385] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:31.575014 25937 raft_consensus.cc:740] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73415f06b3c34da8b3241cb15712dde8, State: Initialized, Role: FOLLOWER
I20260806 07:55:31.575369 26080 log.cc:826] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:31.575601 25937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.576503 26088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.576854 26088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.577682 25937 sys_catalog.cc:565] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:31.578549 26080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: No bootstrap required, opened a new log
W20260806 07:55:31.582821 25891 tablet.cc:2406] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:31.585709 26080 raft_consensus.cc:359] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.586251 26080 raft_consensus.cc:385] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:31.586437 26080 raft_consensus.cc:740] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365ea10e275c44f6b99bbbd1dbd97bbc, State: Initialized, Role: FOLLOWER
I20260806 07:55:31.587203 26080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.588948 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.589236 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.590085 26080 sys_catalog.cc:565] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:31.596524 26100 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:31.596712 26100 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:31.595126 25871 master_runner.cc:187] Error getting master registration for 127.20.202.253:34379: OK, instance_id { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" instance_seqno: 1786002931046224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:31.601126 25871 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:55:31.620551 26112 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:31.620903 26112 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:31.620846 26012 master_runner.cc:187] Error getting master registration for 127.20.202.253:34379: OK, instance_id { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" instance_seqno: 1786002931046224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:31.627455 26012 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:31.753410 26009 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } attempt: 5
W20260806 07:55:31.755522 26034 tablet.cc:2406] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:31.764680 26009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: Bootstrap starting.
I20260806 07:55:31.768198 26009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:31.769289 26009 log.cc:826] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:31.771845 26009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: No bootstrap required, opened a new log
I20260806 07:55:31.776441 26099 raft_consensus.cc:493] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:55:31.776739 26099 raft_consensus.cc:515] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.777225 26009 raft_consensus.cc:359] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.777619 26009 raft_consensus.cc:385] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:31.777793 26009 raft_consensus.cc:740] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c12f187e7f43ffacab7814ad6b2903, State: Initialized, Role: FOLLOWER
I20260806 07:55:31.778566 26099 leader_election.cc:290] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73415f06b3c34da8b3241cb15712dde8 (127.20.202.254:35663), d8c12f187e7f43ffacab7814ad6b2903 (127.20.202.253:34379)
I20260806 07:55:31.778602 26009 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.780161 26115 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.780464 26115 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.780309 25912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73415f06b3c34da8b3241cb15712dde8" is_pre_election: true
I20260806 07:55:31.780807 25912 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365ea10e275c44f6b99bbbd1dbd97bbc in term 0.
I20260806 07:55:31.781622 26029 leader_election.cc:304] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365ea10e275c44f6b99bbbd1dbd97bbc, 73415f06b3c34da8b3241cb15712dde8; no voters:
I20260806 07:55:31.782083 26099 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:55:31.782227 26099 raft_consensus.cc:493] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:55:31.782347 26099 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:31.783365 25984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c12f187e7f43ffacab7814ad6b2903" is_pre_election: true
I20260806 07:55:31.783919 25984 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365ea10e275c44f6b99bbbd1dbd97bbc in term 0.
I20260806 07:55:31.784309 26009 sys_catalog.cc:565] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:31.785876 25962 tablet.cc:2406] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:31.788166 26099 raft_consensus.cc:515] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.790120 25912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73415f06b3c34da8b3241cb15712dde8"
I20260806 07:55:31.790341 25912 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:31.794384 25984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c12f187e7f43ffacab7814ad6b2903"
I20260806 07:55:31.794566 25984 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:31.797983 25984 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365ea10e275c44f6b99bbbd1dbd97bbc in term 1.
I20260806 07:55:31.798524 26032 leader_election.cc:304] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365ea10e275c44f6b99bbbd1dbd97bbc, d8c12f187e7f43ffacab7814ad6b2903; no voters:
I20260806 07:55:31.799546 26099 leader_election.cc:290] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [CANDIDATE]: Term 1 election: Requested vote from peers 73415f06b3c34da8b3241cb15712dde8 (127.20.202.254:35663), d8c12f187e7f43ffacab7814ad6b2903 (127.20.202.253:34379)
I20260806 07:55:31.799973 26099 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:55:31.800516 25912 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365ea10e275c44f6b99bbbd1dbd97bbc in term 1.
I20260806 07:55:31.800825 26099 raft_consensus.cc:697] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 1 LEADER]: Becoming Leader. State: Replica: 365ea10e275c44f6b99bbbd1dbd97bbc, State: Running, Role: LEADER
I20260806 07:55:31.802712 26099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } }
I20260806 07:55:31.820897 26118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365ea10e275c44f6b99bbbd1dbd97bbc. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.821409 26118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:31.828794 26124 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:55:31.831328 26124 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:55:31.843909 25984 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 1 FOLLOWER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:31.845090 26118 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:31.852926 25912 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:31.853966 26118 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:31.871274 26115 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365ea10e275c44f6b99bbbd1dbd97bbc. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.871544 26115 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.873579 26127 mvcc.cc:204] Tried to move back new op lower bound from 7315468008821067776 to 7315468008674861056. Current Snapshot: MvccSnapshot[applied={T|T < 7315468008821067776}]
I20260806 07:55:31.875581 26130 mvcc.cc:204] Tried to move back new op lower bound from 7315468008821067776 to 7315468008674861056. Current Snapshot: MvccSnapshot[applied={T|T < 7315468008821067776}]
I20260806 07:55:31.882426 26088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365ea10e275c44f6b99bbbd1dbd97bbc. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.882678 26088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.886991 26131 mvcc.cc:204] Tried to move back new op lower bound from 7315468008821067776 to 7315468008674861056. Current Snapshot: MvccSnapshot[applied={T|T < 7315468008821067776}]
I20260806 07:55:31.892671 26088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.892874 26088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.893898 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.894150 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:31.894307 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.894480 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:31.895087 26115 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } }
I20260806 07:55:31.895335 26115 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:31.898005 26124 catalog_manager.cc:1383] Generated new cluster ID: 8521c50665294e5ca087d00699c9aa71
I20260806 07:55:31.898110 26124 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:55:31.902834 26135 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 07:55:31.936637 26124 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:55:31.938457 26124 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:55:31.959985 26124 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: Generated new TSK 0
I20260806 07:55:31.962445 26124 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:55:31.992645 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260806 07:55:32.393779 26141 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:32.394083 26141 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:32.394222 26141 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:32.403015 26141 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:32.403214 26141 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:32.413417 26141 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:32.415109 26141 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:32.416970 26141 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:32.426812 26147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:32.426837 26146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:32.428553 26149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:32.428936 26141 server_base.cc:1061] running on GCE node
I20260806 07:55:32.429553 26141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:32.430699 26141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:32.431907 26141 hybrid_clock.cc:648] HybridClock initialized: now 1786002932431783 us; error 93 us; skew 500 ppm
I20260806 07:55:32.432258 26141 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:32.434566 26141 webserver.cc:533] Webserver started at http://127.20.202.193:32919/ using document root <none> and password file <none>
I20260806 07:55:32.435405 26141 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:32.435541 26141 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:32.435994 26141 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:32.438665 26141 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:32.439492 26141 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:32.444968 26141 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:55:32.448431 26155 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:32.449945 26141 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:55:32.450136 26141 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:32.450361 26141 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:32.469785 26141 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:32.470644 26141 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:32.470932 26141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:32.471902 26141 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:32.473573 26141 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:32.473766 26141 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:55:32.473888 26141 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:32.473976 26141 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:32.509842 26141 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43801
I20260806 07:55:32.509929 26267 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43801 every 8 connection(s)
I20260806 07:55:32.511337 26141 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:32.520277 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26141
I20260806 07:55:32.520517 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:55:32.598969 26100 catalog_manager.cc:1295] Loaded cluster ID: 8521c50665294e5ca087d00699c9aa71
I20260806 07:55:32.599078 26100 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: loading cluster ID for follower catalog manager: success
I20260806 07:55:32.601785 26270 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34379
I20260806 07:55:32.602210 26270 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:32.603303 26270 heartbeater.cc:507] Master 127.20.202.253:34379 requested a full tablet report, sending...
I20260806 07:55:32.605096 26100 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: acquiring CA information for follower catalog manager: success
I20260806 07:55:32.606021 25973 ts_manager.cc:194] Registered new tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:32.607641 26268 heartbeater.cc:344] Connected to a master server at 127.20.202.252:39135
I20260806 07:55:32.607771 26268 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:32.608281 26268 heartbeater.cc:507] Master 127.20.202.252:39135 requested a full tablet report, sending...
I20260806 07:55:32.608404 26100 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:32.609608 26269 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35663
I20260806 07:55:32.609750 26269 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:32.610093 26045 ts_manager.cc:194] Registered new tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:32.610226 26269 heartbeater.cc:507] Master 127.20.202.254:35663 requested a full tablet report, sending...
I20260806 07:55:32.611934 25902 ts_manager.cc:194] Registered new tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:32.612251 26045 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:57699
I20260806 07:55:32.623556 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:32.626528 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:36919
--webserver_interface=127.20.202.251
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919 with env {}
I20260806 07:55:32.907934 26135 catalog_manager.cc:1295] Loaded cluster ID: 8521c50665294e5ca087d00699c9aa71
I20260806 07:55:32.908089 26135 catalog_manager.cc:1588] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: loading cluster ID for follower catalog manager: success
I20260806 07:55:32.912245 26135 catalog_manager.cc:1610] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: acquiring CA information for follower catalog manager: success
I20260806 07:55:32.914858 26135 catalog_manager.cc:1638] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:55:32.979171 26278 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:32.979534 26278 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:32.979666 26278 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:32.988314 26278 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:32.988435 26278 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:32.988555 26278 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:32.988636 26278 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:32.999155 26278 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:36919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:33.000857 26278 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:33.002830 26278 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:33.012979 26285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:33.012946 26284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:33.019238 26287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:33.019771 26278 server_base.cc:1061] running on GCE node
I20260806 07:55:33.020828 26278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:33.023231 26278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:33.024555 26278 hybrid_clock.cc:648] HybridClock initialized: now 1786002933024361 us; error 157 us; skew 500 ppm
I20260806 07:55:33.025000 26278 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:33.027895 26278 webserver.cc:533] Webserver started at http://127.20.202.251:33897/ using document root <none> and password file <none>
I20260806 07:55:33.028911 26278 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:33.029035 26278 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:33.029451 26278 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:33.032792 26278 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "27b4a7b58e194987a50fe5b331646609"
format_stamp: "Formatted at 2026-08-06 07:55:33 on dist-test-slave-g350"
I20260806 07:55:33.033875 26278 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "27b4a7b58e194987a50fe5b331646609"
format_stamp: "Formatted at 2026-08-06 07:55:33 on dist-test-slave-g350"
I20260806 07:55:33.041232 26278 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 07:55:33.045794 26294 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:33.047489 26278 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:33.047664 26278 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "27b4a7b58e194987a50fe5b331646609"
format_stamp: "Formatted at 2026-08-06 07:55:33 on dist-test-slave-g350"
I20260806 07:55:33.047904 26278 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:33.074873 26278 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:33.076063 26278 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:33.076392 26278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:33.100687 26345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:36919 every 8 connection(s)
I20260806 07:55:33.100723 26278 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:36919
I20260806 07:55:33.102860 26278 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:33.107585 26346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:33.108230 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26278
I20260806 07:55:33.117200 26346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:33.142362 26346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:33.149708 26346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:33.156420 26346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:33.166468 26346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Bootstrap starting.
I20260806 07:55:33.169524 26346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:33.170611 26346 log.cc:826] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:33.173085 26346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: No bootstrap required, opened a new log
I20260806 07:55:33.178108 26346 raft_consensus.cc:359] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } }
I20260806 07:55:33.178449 26346 raft_consensus.cc:385] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:33.178561 26346 raft_consensus.cc:740] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27b4a7b58e194987a50fe5b331646609, State: Initialized, Role: FOLLOWER
I20260806 07:55:33.179235 26346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } }
I20260806 07:55:33.180248 26355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } } }
I20260806 07:55:33.180567 26355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:33.181514 26346 sys_catalog.cc:565] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:33.192457 26366 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:33.192690 26366 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:33.199709 26278 master_runner.cc:429] Detected that this master 27b4a7b58e194987a50fe5b331646609 is joining an existing cluster
I20260806 07:55:33.199826 26278 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:55:33.200752 26278 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:55:33.201093 26278 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:55:33.201327 26278 tablet_replica.cc:333] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: stopping tablet replica
I20260806 07:55:33.202951 26278 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:36919
I20260806 07:55:33.255772 26045 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:36919: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "27b4a7b58e194987a50fe5b331646609"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 36919
}
attrs {
promote: true
}
}
dest_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc"
cas_config_opid_index: -1
I20260806 07:55:33.257269 26045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } }
I20260806 07:55:33.261039 25911 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:55:33.261248 26029 consensus_peers.cc:597] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc -> Peer 27b4a7b58e194987a50fe5b331646609 (127.20.202.251:36919): Couldn't send request to peer 27b4a7b58e194987a50fe5b331646609. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:33.261947 26099 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:33.263479 25983 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 1 FOLLOWER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:55:33.264310 26099 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:33.280651 26377 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.281237 25911 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.286055 26380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.286443 26380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:33.286576 25983 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.289789 26377 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:36919
I20260806 07:55:33.290562 26278 master.cc:562] Master@127.20.202.251:36919 shutting down...
I20260806 07:55:33.308755 26382 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.309298 26382 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:33.310900 26278 master.cc:584] Master@127.20.202.251:36919 shutdown complete.
I20260806 07:55:33.311115 26278 master_runner.cc:306] Clearing existing system tablet
I20260806 07:55:33.318130 26278 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:33.318362 26278 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:33.326391 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.326684 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:33.330381 26278 fs_manager.cc:714] Time spent opening directory manager: real 0.011s user 0.005s sys 0.001s
I20260806 07:55:33.330701 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.331138 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:33.331548 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.331748 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:33.332391 26387 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:33.333926 26278 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:33.334220 26278 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:55:33.334347 26278 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "27b4a7b58e194987a50fe5b331646609"
format_stamp: "Formatted at 2026-08-06 07:55:33 on dist-test-slave-g350"
I20260806 07:55:33.340008 26278 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:55:33.347613 26278 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:55:33.347795 26278 log.cc:1199] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:55:33.348403 26278 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Deleting consensus metadata
I20260806 07:55:33.348716 26278 master_runner.cc:316] Copying system tablet from 127.20.202.252:39135
I20260806 07:55:33.351886 26278 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.252:39135
I20260806 07:55:33.369083 26065 tablet_copy_service.cc:140] P 365ea10e275c44f6b99bbbd1dbd97bbc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 27b4a7b58e194987a50fe5b331646609 ({username='slave'} at 127.0.0.1:40754)
I20260806 07:55:33.369400 26065 tablet_copy_service.cc:161] P 365ea10e275c44f6b99bbbd1dbd97bbc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 27b4a7b58e194987a50fe5b331646609 at {username='slave'} at 127.0.0.1:40754: session id = 27b4a7b58e194987a50fe5b331646609-00000000000000000000000000000000
I20260806 07:55:33.373924 26065 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:55:33.376514 26278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:55:33.384019 26278 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: tablet copy: Starting download of 0 data blocks...
I20260806 07:55:33.384425 26278 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: tablet copy: Starting download of 1 WAL segments...
I20260806 07:55:33.387524 26278 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:55:33.392297 26065 tablet_copy_service.cc:342] P 365ea10e275c44f6b99bbbd1dbd97bbc: Request end of tablet copy session 27b4a7b58e194987a50fe5b331646609-00000000000000000000000000000000 received from {username='slave'} at 127.0.0.1:40754
I20260806 07:55:33.392563 26065 tablet_copy_service.cc:434] P 365ea10e275c44f6b99bbbd1dbd97bbc: ending tablet copy session 27b4a7b58e194987a50fe5b331646609-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 27b4a7b58e194987a50fe5b331646609
W20260806 07:55:33.399974 26278 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:55:33.404300 26278 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:33.406991 26394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:33.411000 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:33.418540 26278 server_base.cc:1061] running on GCE node
W20260806 07:55:33.418856 26397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:33.419337 26278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:33.420290 26278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:33.421559 26278 hybrid_clock.cc:648] HybridClock initialized: now 1786002933421400 us; error 97 us; skew 500 ppm
I20260806 07:55:33.421869 26278 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:33.423426 26278 webserver.cc:533] Webserver started at http://127.20.202.251:46229/ using document root <none> and password file <none>
I20260806 07:55:33.423832 26278 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:33.423935 26278 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:33.426905 26278 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 07:55:33.428884 26403 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:33.429358 26278 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 07:55:33.429488 26278 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "27b4a7b58e194987a50fe5b331646609"
format_stamp: "Formatted at 2026-08-06 07:55:33 on dist-test-slave-g350"
I20260806 07:55:33.429669 26278 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:33.443946 26278 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:33.444859 26278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:33.464265 26278 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:36919
I20260806 07:55:33.464303 26454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:36919 every 8 connection(s)
I20260806 07:55:33.465433 26278 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:33.469031 26455 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:33.471376 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Bootstrap starting.
I20260806 07:55:33.484127 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:33.484571 26455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: Bootstrap complete.
I20260806 07:55:33.485669 26455 raft_consensus.cc:359] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } }
I20260806 07:55:33.485939 26455 raft_consensus.cc:740] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27b4a7b58e194987a50fe5b331646609, State: Initialized, Role: LEARNER
I20260806 07:55:33.486212 26455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } }
I20260806 07:55:33.486984 26458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.487273 26458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:33.487308 26455 sys_catalog.cc:565] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:33.495805 26469 catalog_manager.cc:1295] Loaded cluster ID: 8521c50665294e5ca087d00699c9aa71
I20260806 07:55:33.495942 26469 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: loading cluster ID for follower catalog manager: success
I20260806 07:55:33.499919 26469 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: acquiring CA information for follower catalog manager: success
I20260806 07:55:33.502683 26469 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:33.503643 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26141
I20260806 07:55:33.514871 26263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:33.649121 26268 heartbeater.cc:499] Master 127.20.202.252:39135 was elected leader, sending a full tablet report...
I20260806 07:55:33.654153 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26141
I20260806 07:55:33.673429 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43801
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--webserver_port=32919
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919 with env {}
I20260806 07:55:33.725054 26430 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:55:33.725841 26458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365ea10e275c44f6b99bbbd1dbd97bbc. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:33.726089 26458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:55:34.036487 26473 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:34.036850 26473 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:34.037001 26473 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:34.045653 26473 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:34.045873 26473 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:34.059319 26473 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43801
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=32919
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:34.061378 26473 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:34.063743 26473 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:34.074860 26478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:34.074826 26479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:34.075559 26481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:34.075982 26473 server_base.cc:1061] running on GCE node
I20260806 07:55:34.076563 26473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:34.077653 26473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:34.078922 26473 hybrid_clock.cc:648] HybridClock initialized: now 1786002934078787 us; error 101 us; skew 500 ppm
I20260806 07:55:34.079283 26473 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:34.081467 26473 webserver.cc:533] Webserver started at http://127.20.202.193:32919/ using document root <none> and password file <none>
I20260806 07:55:34.082309 26473 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:34.082469 26473 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:34.088300 26473 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 07:55:34.091156 26487 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:34.092568 26473 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:34.092760 26473 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:34.093442 26473 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:34.127419 26473 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:34.128280 26473 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:34.128532 26473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:34.129465 26473 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:34.131114 26473 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:34.131186 26473 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:34.131316 26473 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:34.131387 26473 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:34.166738 26473 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43801
I20260806 07:55:34.166829 26600 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43801 every 8 connection(s)
I20260806 07:55:34.168316 26473 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:34.168853 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26473
I20260806 07:55:34.194299 26603 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36919
I20260806 07:55:34.194278 26377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: true } } }
I20260806 07:55:34.194607 26377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:34.194729 26603 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:34.196012 26603 heartbeater.cc:507] Master 127.20.202.251:36919 requested a full tablet report, sending...
I20260806 07:55:34.200688 26420 ts_manager.cc:194] Registered new tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:34.203383 26613 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc: attempting to promote NON_VOTER 27b4a7b58e194987a50fe5b331646609 to VOTER
I20260806 07:55:34.204463 26613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } }
I20260806 07:55:34.204843 26601 heartbeater.cc:344] Connected to a master server at 127.20.202.252:39135
I20260806 07:55:34.204968 26601 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:34.205477 26601 heartbeater.cc:507] Master 127.20.202.252:39135 requested a full tablet report, sending...
I20260806 07:55:34.206794 26045 ts_manager.cc:194] Re-registered known tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:34.207505 25983 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 1 FOLLOWER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:34.207633 26430 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 1 LEARNER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:34.207818 26045 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:60251
I20260806 07:55:34.208259 25911 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Refusing update from remote peer 365ea10e275c44f6b99bbbd1dbd97bbc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:34.208328 26614 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:34.209518 26377 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:34.210168 26613 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:34.210824 26602 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35663
I20260806 07:55:34.211023 26602 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:34.211431 26602 heartbeater.cc:507] Master 127.20.202.254:35663 requested a full tablet report, sending...
I20260806 07:55:34.213052 25902 ts_manager.cc:194] Re-registered known tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:34.213229 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:34.213964 26377 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.216504 26604 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34379
I20260806 07:55:34.216667 26604 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:34.216960 26604 heartbeater.cc:507] Master 127.20.202.253:34379 requested a full tablet report, sending...
I20260806 07:55:34.217669 25911 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.218163 26377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.218315 26377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:34.218410 25973 ts_manager.cc:194] Re-registered known tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:34.219269 25983 raft_consensus.cc:2955] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.221853 26613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.222106 26613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:34.222800 26613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.223088 26613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:34.230384 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:34.230655 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25871
I20260806 07:55:34.231920 25932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:34.232699 26617 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.232955 26617 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c12f187e7f43ffacab7814ad6b2903 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:34.233443 26614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.233724 26614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:34.234496 26430 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 27b4a7b58e194987a50fe5b331646609 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.239498 26621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.239858 26621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b4a7b58e194987a50fe5b331646609 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:34.345124 26619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.345467 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25871
I20260806 07:55:34.370987 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35663
--webserver_interface=127.20.202.254
--webserver_port=44649
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919 with env {}
W20260806 07:55:34.637111 26029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:35663: connect: Connection refused (error 111)
W20260806 07:55:34.652474 26029 consensus_peers.cc:597] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc -> Peer 73415f06b3c34da8b3241cb15712dde8 (127.20.202.254:35663): Couldn't send request to peer 73415f06b3c34da8b3241cb15712dde8. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:35663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:55:34.731827 26622 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:34.732226 26622 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:34.732364 26622 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:34.741218 26622 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:34.741356 26622 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:34.741423 26622 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:34.741475 26622 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:34.751904 26622 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=44649
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:34.753572 26622 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:34.755496 26622 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:34.763492 26630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:34.763782 26631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:34.765100 26633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:34.765471 26622 server_base.cc:1061] running on GCE node
I20260806 07:55:34.766322 26622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:34.767894 26622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:34.769166 26622 hybrid_clock.cc:648] HybridClock initialized: now 1786002934769015 us; error 117 us; skew 500 ppm
I20260806 07:55:34.769567 26622 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:34.771961 26622 webserver.cc:533] Webserver started at http://127.20.202.254:44649/ using document root <none> and password file <none>
I20260806 07:55:34.772708 26622 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:34.772861 26622 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:34.778267 26622 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:55:34.781193 26639 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:34.782598 26622 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 07:55:34.782773 26622 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "73415f06b3c34da8b3241cb15712dde8"
format_stamp: "Formatted at 2026-08-06 07:55:30 on dist-test-slave-g350"
I20260806 07:55:34.783528 26622 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:34.805446 26622 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:34.806445 26622 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:34.806777 26622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:34.828161 26622 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:35663
I20260806 07:55:34.828188 26690 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:35663 every 8 connection(s)
I20260806 07:55:34.830006 26622 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:34.833267 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26622
I20260806 07:55:34.833568 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26473
I20260806 07:55:34.838356 26691 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:34.843369 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Bootstrap starting.
I20260806 07:55:34.844202 26596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:34.867969 26691 log.cc:826] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:34.880084 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:34.880749 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: Bootstrap complete.
I20260806 07:55:34.891059 26691 raft_consensus.cc:359] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } }
I20260806 07:55:34.892040 26691 raft_consensus.cc:740] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73415f06b3c34da8b3241cb15712dde8, State: Initialized, Role: FOLLOWER
I20260806 07:55:34.892930 26691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } }
I20260806 07:55:34.893898 26697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:34.894214 26697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:34.895191 26691 sys_catalog.cc:565] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:34.921363 26709 catalog_manager.cc:1295] Loaded cluster ID: 8521c50665294e5ca087d00699c9aa71
I20260806 07:55:34.921571 26709 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: loading cluster ID for follower catalog manager: success
I20260806 07:55:34.926082 26709 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: acquiring CA information for follower catalog manager: success
I20260806 07:55:34.928936 26709 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:34.971043 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26473
I20260806 07:55:34.997073 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43801
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--webserver_port=32919
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919 with env {}
I20260806 07:55:35.018527 26614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:35.018853 26614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:35.029805 26697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365ea10e275c44f6b99bbbd1dbd97bbc. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:35.030054 26697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73415f06b3c34da8b3241cb15712dde8 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:55:35.381592 26716 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:35.382072 26716 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:35.382177 26716 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:35.395893 26716 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:35.396162 26716 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:35.409303 26716 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43801
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=32919
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:35.411046 26716 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:35.412957 26716 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:35.424141 26726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:35.424234 26728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:35.424180 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:35.425168 26716 server_base.cc:1061] running on GCE node
I20260806 07:55:35.425835 26716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:35.426947 26716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:35.428190 26716 hybrid_clock.cc:648] HybridClock initialized: now 1786002935428047 us; error 114 us; skew 500 ppm
I20260806 07:55:35.428514 26716 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:35.430783 26716 webserver.cc:533] Webserver started at http://127.20.202.193:32919/ using document root <none> and password file <none>
I20260806 07:55:35.431595 26716 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:35.431772 26716 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:35.437315 26716 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:55:35.440357 26734 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:35.441742 26716 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:35.441916 26716 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:35.442653 26716 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:35.475270 26716 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:35.476214 26716 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:35.476521 26716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:35.477680 26716 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:35.479671 26716 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:35.479744 26716 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:35.479830 26716 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:35.479873 26716 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:35.486548 26718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "73415f06b3c34da8b3241cb15712dde8" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35663 } } peers { permanent_uuid: "d8c12f187e7f43ffacab7814ad6b2903" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34379 } } peers { permanent_uuid: "365ea10e275c44f6b99bbbd1dbd97bbc" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 39135 } } peers { permanent_uuid: "27b4a7b58e194987a50fe5b331646609" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36919 } attrs { promote: false } } }
I20260806 07:55:35.486907 26718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365ea10e275c44f6b99bbbd1dbd97bbc [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:35.519604 26716 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43801
I20260806 07:55:35.519695 26847 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43801 every 8 connection(s)
I20260806 07:55:35.521224 26716 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:35.529841 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26716
I20260806 07:55:35.568606 26849 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35663
I20260806 07:55:35.569075 26849 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:35.570879 26849 heartbeater.cc:507] Master 127.20.202.254:35663 requested a full tablet report, sending...
I20260806 07:55:35.572178 26848 heartbeater.cc:344] Connected to a master server at 127.20.202.252:39135
I20260806 07:55:35.572320 26848 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:35.572818 26848 heartbeater.cc:507] Master 127.20.202.252:39135 requested a full tablet report, sending...
I20260806 07:55:35.574018 26045 ts_manager.cc:194] Re-registered known tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:35.575219 26045 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:42001
I20260806 07:55:35.577003 26656 ts_manager.cc:194] Registered new tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:35.578317 26850 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36919
I20260806 07:55:35.578478 26850 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:35.578840 26850 heartbeater.cc:507] Master 127.20.202.251:36919 requested a full tablet report, sending...
I20260806 07:55:35.580267 26420 ts_manager.cc:194] Re-registered known tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:35.582000 26851 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34379
I20260806 07:55:35.582175 26851 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:35.582439 26851 heartbeater.cc:507] Master 127.20.202.253:34379 requested a full tablet report, sending...
I20260806 07:55:35.583887 25973 ts_manager.cc:194] Re-registered known tserver with Master: 1c5f1142fddc490e89f4f106a5ab2983 (127.20.202.193:43801)
I20260806 07:55:35.614579 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26716
I20260806 07:55:35.626914 26843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:35.717377 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26716
I20260806 07:55:35.737305 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43801
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--webserver_port=32919
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--fault_before_start=1.0 with env {}
W20260806 07:55:36.096292 26865 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:36.096601 26865 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:36.096731 26865 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:55:36.096810 26865 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:36.105257 26865 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:36.105439 26865 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:36.115899 26865 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43801
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=32919
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:36.117703 26865 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:36.119794 26865 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:36.129915 26871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:36.129964 26870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:36.129984 26873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:36.131316 26865 server_base.cc:1061] running on GCE node
I20260806 07:55:36.131888 26865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:36.132959 26865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:36.134251 26865 hybrid_clock.cc:648] HybridClock initialized: now 1786002936134074 us; error 127 us; skew 500 ppm
I20260806 07:55:36.134548 26865 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:36.136725 26865 webserver.cc:533] Webserver started at http://127.20.202.193:32919/ using document root <none> and password file <none>
I20260806 07:55:36.137451 26865 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:36.137650 26865 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:36.142982 26865 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 07:55:36.145665 26879 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:36.147286 26865 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:55:36.147500 26865 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:36.148378 26865 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:36.174088 26865 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:36.175166 26865 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:36.175472 26865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:36.176671 26865 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:36.178571 26865 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:36.178666 26865 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:36.178835 26865 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:36.178946 26865 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:55:36.179361 26865 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:55:36.179476 26865 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:55:36.179584 26865 fault_injection.cc:59] injecting fault for kudu (pid 26865): FLAGS_fault_before_start (process will exit)
I20260806 07:55:36.200110 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43801
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135
--webserver_port=32919
--webserver_interface=127.20.202.193
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919 with env {}
W20260806 07:55:36.559083 26887 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:36.559393 26887 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:36.559546 26887 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:36.568220 26887 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:36.568430 26887 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:36.578760 26887 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35917,127.20.202.213:41961,127.20.202.214:35605,127.20.202.215:41627,127.20.202.216:33061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43801
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=32919
--tserver_master_addrs=127.20.202.254:35663,127.20.202.253:34379,127.20.202.252:39135,127.20.202.251:36919
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:36.580467 26887 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:36.582459 26887 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:36.592439 26892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:36.592454 26893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:36.593364 26895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:36.594163 26887 server_base.cc:1061] running on GCE node
I20260806 07:55:36.594681 26887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:36.595741 26887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:36.596938 26887 hybrid_clock.cc:648] HybridClock initialized: now 1786002936596763 us; error 143 us; skew 500 ppm
I20260806 07:55:36.597271 26887 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:55:36.599673 26887 webserver.cc:533] Webserver started at http://127.20.202.193:32919/ using document root <none> and password file <none>
I20260806 07:55:36.600426 26887 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:36.600546 26887 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:36.607190 26887 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 07:55:36.609938 26902 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:36.611268 26887 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:55:36.611441 26887 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "1c5f1142fddc490e89f4f106a5ab2983"
format_stamp: "Formatted at 2026-08-06 07:55:32 on dist-test-slave-g350"
I20260806 07:55:36.612190 26887 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:36.652258 26887 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:36.653179 26887 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:36.653486 26887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:36.654512 26887 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:36.656241 26887 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:36.656313 26887 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:36.656458 26887 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:36.656520 26887 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:36.691825 26887 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43801
I20260806 07:55:36.691886 27015 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43801 every 8 connection(s)
I20260806 07:55:36.693380 26887 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:36.703094 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 26887
I20260806 07:55:36.703441 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26887
I20260806 07:55:36.732424 27016 heartbeater.cc:344] Connected to a master server at 127.20.202.252:39135
I20260806 07:55:36.732915 27011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:36.732920 27016 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:36.821781 27017 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35663
I20260806 07:55:36.822208 27018 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36919
I20260806 07:55:36.822213 27019 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34379
I20260806 07:55:36.822516 27018 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:36.822674 27017 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:36.823098 27019 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:36.823774 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26887
I20260806 07:55:36.839428 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26622
I20260806 07:55:36.846490 26686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:36.952057 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26622
I20260806 07:55:36.975265 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 25941
I20260806 07:55:36.976439 26004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:37.084878 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 25941
I20260806 07:55:37.104326 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26012
I20260806 07:55:37.105576 26075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:37.218421 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26012
I20260806 07:55:37.240739 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 26278
I20260806 07:55:37.242075 26450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:37.352015 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 26278
Shutting down hive metastore.
2026-08-06T07:55:37Z chronyd exiting
2026-08-06T07:55:37Z chronyd exiting
2026-08-06T07:55:37Z chronyd exiting
2026-08-06T07:55:37Z chronyd exiting
2026-08-06T07:55:37Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/5 (23828 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/6
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:55:37 dist-test-slave-g350 krb5kdc[27044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:55:37 dist-test-slave-g350 krb5kdc[27044](info): set up 2 sockets
Aug 06 07:55:37 dist-test-slave-g350 krb5kdc[27044](info): commencing operation
krb5kdc: starting...
W20260806 07:55:39.929878 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.107s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:55:40 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:40.267292 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46431
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:40.617287 27058 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:40.617666 27058 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:40.617844 27058 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:40.627841 27058 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:40.627947 27058 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:40.628149 27058 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:40.628218 27058 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:40.639242 27058 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:40.640797 27058 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:40.642175 27058 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:40.650442 27064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:40.650487 27063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:40.651187 27067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:40.651742 27058 server_base.cc:1061] running on GCE node
I20260806 07:55:40.652447 27058 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:40.652607 27058 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:40.652676 27058 hybrid_clock.cc:648] HybridClock initialized: now 1786002940652659 us; error 0 us; skew 500 ppm
I20260806 07:55:40.652976 27058 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:40 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:40.656343 27058 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:55:40.658402 27058 webserver.cc:533] Webserver started at http://127.20.202.254:37983/ using document root <none> and password file <none>
I20260806 07:55:40.659086 27058 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:40.659183 27058 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:40.659468 27058 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:40.661844 27058 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "f830aa49ae2c4b3287ff111892b706d6"
format_stamp: "Formatted at 2026-08-06 07:55:40 on dist-test-slave-g350"
I20260806 07:55:40.662587 27058 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "f830aa49ae2c4b3287ff111892b706d6"
format_stamp: "Formatted at 2026-08-06 07:55:40 on dist-test-slave-g350"
I20260806 07:55:40.668282 27058 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:55:40.671739 27073 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:40.673156 27058 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:55:40.673344 27058 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "f830aa49ae2c4b3287ff111892b706d6"
format_stamp: "Formatted at 2026-08-06 07:55:40 on dist-test-slave-g350"
I20260806 07:55:40.673553 27058 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:40.689738 27058 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:40.693401 27058 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:40.693722 27058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:40.713816 27058 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:46431
I20260806 07:55:40.713836 27125 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:46431 every 8 connection(s)
I20260806 07:55:40.715433 27058 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:40.718248 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27058
I20260806 07:55:40.718468 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:55:40.719834 27126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:40.726951 27126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:40 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
I20260806 07:55:40.753090 27126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:40.754472 27076 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:42153: connect: Connection refused (error 111)
W20260806 07:55:40.756258 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:42153: Network error: Client connection negotiation failed: client connection to 127.20.202.253:42153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:40.805752 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } attempt: 1
W20260806 07:55:40.808104 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:42153: Network error: Client connection negotiation failed: client connection to 127.20.202.253:42153: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:40.857277 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:42153
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:40.876431 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } attempt: 2
W20260806 07:55:40.878749 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:42153: Network error: Client connection negotiation failed: client connection to 127.20.202.253:42153: connect: Connection refused (error 111)
I20260806 07:55:40.970028 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } attempt: 3
W20260806 07:55:40.972440 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:42153: Network error: Client connection negotiation failed: client connection to 127.20.202.253:42153: connect: Connection refused (error 111)
I20260806 07:55:41.115748 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } attempt: 4
W20260806 07:55:41.118096 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:42153: Network error: Client connection negotiation failed: client connection to 127.20.202.253:42153: connect: Connection refused (error 111)
W20260806 07:55:41.216500 27135 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:41.216892 27135 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:41.217056 27135 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:41.225481 27135 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:41.225648 27135 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:41.225776 27135 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:41.225860 27135 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:41.236711 27135 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:41.238535 27135 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:41.240000 27135 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:41.248914 27140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:41.249668 27141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:41.249922 27143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:41.250581 27135 server_base.cc:1061] running on GCE node
I20260806 07:55:41.251331 27135 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:41.251474 27135 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:41.251545 27135 hybrid_clock.cc:648] HybridClock initialized: now 1786002941251527 us; error 0 us; skew 500 ppm
I20260806 07:55:41.251818 27135 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:41.255330 27135 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:55:41.257436 27135 webserver.cc:533] Webserver started at http://127.20.202.253:38743/ using document root <none> and password file <none>
I20260806 07:55:41.258241 27135 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:41.258342 27135 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:41.258646 27135 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:41.261212 27135 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "e555fb35a6544a1999855b330e894575"
format_stamp: "Formatted at 2026-08-06 07:55:41 on dist-test-slave-g350"
I20260806 07:55:41.262048 27135 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "e555fb35a6544a1999855b330e894575"
format_stamp: "Formatted at 2026-08-06 07:55:41 on dist-test-slave-g350"
I20260806 07:55:41.267620 27135 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:55:41.271283 27149 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:41.272796 27135 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:55:41.272944 27135 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "e555fb35a6544a1999855b330e894575"
format_stamp: "Formatted at 2026-08-06 07:55:41 on dist-test-slave-g350"
I20260806 07:55:41.273128 27135 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:41.294742 27135 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:41.298945 27135 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:41.299217 27135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:41.325208 27135 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:42153
I20260806 07:55:41.325227 27200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:42153 every 8 connection(s)
I20260806 07:55:41.326936 27135 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:55:41.331604 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:41.331773 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27135
I20260806 07:55:41.331983 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:55:41.338518 27201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:41.362418 27201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:41.380467 27201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:41.381970 27150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
W20260806 07:55:41.383858 27201 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:41.417369 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } attempt: 5
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:41.431281 27126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:55:41.433198 27201 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 1
W20260806 07:55:41.433502 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
W20260806 07:55:41.435278 27201 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:41.475138 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36857
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:41.484784 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 1
W20260806 07:55:41.487273 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
I20260806 07:55:41.503525 27201 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 2
W20260806 07:55:41.505759 27201 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
I20260806 07:55:41.555593 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 2
W20260806 07:55:41.558091 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
I20260806 07:55:41.597065 27201 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 3
W20260806 07:55:41.599524 27201 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
I20260806 07:55:41.664387 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 3
W20260806 07:55:41.666792 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
I20260806 07:55:41.742803 27201 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 4
W20260806 07:55:41.745026 27201 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
W20260806 07:55:41.830976 27212 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:41.831363 27212 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:41.831526 27212 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:41.840054 27212 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:41.840173 27212 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:41.840276 27212 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:41.840376 27212 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:41.844053 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 4
W20260806 07:55:41.846292 27126 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:36857: Network error: Client connection negotiation failed: client connection to 127.20.202.252:36857: connect: Connection refused (error 111)
I20260806 07:55:41.852335 27212 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:36857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:41.854028 27212 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:41.855377 27212 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:41.863855 27217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:41.863899 27218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:41.864816 27212 server_base.cc:1061] running on GCE node
W20260806 07:55:41.864952 27220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:41.865837 27212 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:41.866025 27212 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:41.866101 27212 hybrid_clock.cc:648] HybridClock initialized: now 1786002941866082 us; error 0 us; skew 500 ppm
I20260806 07:55:41.866464 27212 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:41.869845 27212 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:55:41.872232 27212 webserver.cc:533] Webserver started at http://127.20.202.252:34227/ using document root <none> and password file <none>
I20260806 07:55:41.872961 27212 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:41.873059 27212 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:41.873368 27212 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:41.875787 27212 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "37dc325715bd44f6bd9f1fabb76a2e4a"
format_stamp: "Formatted at 2026-08-06 07:55:41 on dist-test-slave-g350"
I20260806 07:55:41.876506 27212 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "37dc325715bd44f6bd9f1fabb76a2e4a"
format_stamp: "Formatted at 2026-08-06 07:55:41 on dist-test-slave-g350"
I20260806 07:55:41.882534 27212 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260806 07:55:41.886220 27226 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:41.887674 27212 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:55:41.887879 27212 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "37dc325715bd44f6bd9f1fabb76a2e4a"
format_stamp: "Formatted at 2026-08-06 07:55:41 on dist-test-slave-g350"
I20260806 07:55:41.888090 27212 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:41.907490 27212 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:41.914778 27212 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:41.915098 27212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:41.937579 27212 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:36857
I20260806 07:55:41.937712 27277 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:36857 every 8 connection(s)
I20260806 07:55:41.939386 27212 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:55:41.945741 27278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:41.947337 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27212
I20260806 07:55:41.947540 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:55:41.954977 27278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:41.989378 27278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:55:41 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:42.005261 27278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:42 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:42.023533 27278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: Bootstrap starting.
I20260806 07:55:42.027220 27278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:42.028338 27278 log.cc:826] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:42.031255 27278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: No bootstrap required, opened a new log
I20260806 07:55:42.037036 27278 raft_consensus.cc:359] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.037335 27278 raft_consensus.cc:385] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:42.037480 27278 raft_consensus.cc:740] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37dc325715bd44f6bd9f1fabb76a2e4a, State: Initialized, Role: FOLLOWER
I20260806 07:55:42.038162 27278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.039135 27287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.039412 27287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.040222 27278 sys_catalog.cc:565] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:42.044289 27201 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 5
W20260806 07:55:42.053205 27298 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:42.053447 27298 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 07:55:42 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:42.051548 27212 master_runner.cc:187] Error getting master registration for 127.20.202.254:46431: OK, instance_id { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" instance_seqno: 1786002940708221 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:42.059971 27212 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:42.066068 27201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: Bootstrap starting.
I20260806 07:55:42.069551 27201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:42.070668 27201 log.cc:826] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:42.073129 27201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: No bootstrap required, opened a new log
I20260806 07:55:42.078294 27201 raft_consensus.cc:359] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.078663 27201 raft_consensus.cc:385] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:42.078806 27201 raft_consensus.cc:740] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e555fb35a6544a1999855b330e894575, State: Initialized, Role: FOLLOWER
I20260806 07:55:42.079499 27201 consensus_queue.cc:260] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.080556 27301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.080969 27301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.082043 27201 sys_catalog.cc:565] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:42.093029 27312 catalog_manager.cc:1594] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:42.093982 27312 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:42.090248 27135 master_runner.cc:187] Error getting master registration for 127.20.202.254:46431: OK, instance_id { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" instance_seqno: 1786002940708221 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:42.099884 27135 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:42.123548 27126 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } attempt: 5
Aug 06 07:55:42 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:42.148059 27126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Bootstrap starting.
I20260806 07:55:42.152771 27126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:42.154407 27126 log.cc:826] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:42.158028 27126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: No bootstrap required, opened a new log
I20260806 07:55:42.166564 27126 raft_consensus.cc:359] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.166967 27126 raft_consensus.cc:385] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:42.167116 27126 raft_consensus.cc:740] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f830aa49ae2c4b3287ff111892b706d6, State: Initialized, Role: FOLLOWER
W20260806 07:55:42.167441 27232 tablet.cc:2406] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:42.168031 27126 consensus_queue.cc:260] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.169198 27315 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.169620 27315 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.170651 27126 sys_catalog.cc:565] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:42.190935 27058 master_runner.cc:264] No leader master found from master f830aa49ae2c4b3287ff111892b706d6
I20260806 07:55:42.191134 27058 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:55:42.192642 27326 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:42.192893 27326 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 07:55:42.196907 27079 tablet.cc:2406] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:55:42.304148 27155 tablet.cc:2406] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:42.446429 27287 raft_consensus.cc:493] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:55:42.446810 27287 raft_consensus.cc:515] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.448998 27287 leader_election.cc:290] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f830aa49ae2c4b3287ff111892b706d6 (127.20.202.254:46431), e555fb35a6544a1999855b330e894575 (127.20.202.253:42153)
I20260806 07:55:42.451205 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e555fb35a6544a1999855b330e894575" is_pre_election: true
I20260806 07:55:42.451753 27176 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37dc325715bd44f6bd9f1fabb76a2e4a in term 0.
I20260806 07:55:42.452889 27229 leader_election.cc:304] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37dc325715bd44f6bd9f1fabb76a2e4a, e555fb35a6544a1999855b330e894575; no voters:
I20260806 07:55:42.454075 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f830aa49ae2c4b3287ff111892b706d6" is_pre_election: true
I20260806 07:55:42.454598 27100 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37dc325715bd44f6bd9f1fabb76a2e4a in term 0.
I20260806 07:55:42.455694 27287 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:55:42.455873 27287 raft_consensus.cc:493] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:55:42.455982 27287 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:42.461488 27287 raft_consensus.cc:515] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.464859 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f830aa49ae2c4b3287ff111892b706d6"
I20260806 07:55:42.465127 27100 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:42.468951 27100 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37dc325715bd44f6bd9f1fabb76a2e4a in term 1.
I20260806 07:55:42.469767 27230 leader_election.cc:304] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37dc325715bd44f6bd9f1fabb76a2e4a, f830aa49ae2c4b3287ff111892b706d6; no voters:
I20260806 07:55:42.478925 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e555fb35a6544a1999855b330e894575"
I20260806 07:55:42.479259 27176 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:42.480621 27328 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:55:42.483960 27176 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37dc325715bd44f6bd9f1fabb76a2e4a in term 1.
I20260806 07:55:42.485169 27287 leader_election.cc:290] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [CANDIDATE]: Term 1 election: Requested vote from peers f830aa49ae2c4b3287ff111892b706d6 (127.20.202.254:46431), e555fb35a6544a1999855b330e894575 (127.20.202.253:42153)
I20260806 07:55:42.488379 27328 raft_consensus.cc:697] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 1 LEADER]: Becoming Leader. State: Replica: 37dc325715bd44f6bd9f1fabb76a2e4a, State: Running, Role: LEADER
I20260806 07:55:42.489166 27328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } }
I20260806 07:55:42.514751 27287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37dc325715bd44f6bd9f1fabb76a2e4a. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.515108 27287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:42.516880 27332 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:55:42.519970 27332 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:55:42.536636 27176 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 1 FOLLOWER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:42.538231 27287 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:42.546451 27100 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:42.549125 27287 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:42.572968 27301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37dc325715bd44f6bd9f1fabb76a2e4a. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.573307 27301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.578169 27333 mvcc.cc:204] Tried to move back new op lower bound from 7315468052614664192 to 7315468052463034368. Current Snapshot: MvccSnapshot[applied={T|T < 7315468052614664192}]
I20260806 07:55:42.578984 27334 mvcc.cc:204] Tried to move back new op lower bound from 7315468052614664192 to 7315468052463034368. Current Snapshot: MvccSnapshot[applied={T|T < 7315468052614664192}]
I20260806 07:55:42.582093 27330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.582424 27330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:42.588842 27301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.589020 27332 catalog_manager.cc:1383] Generated new cluster ID: e3a8337d3f3d44678292dda3271750e8
I20260806 07:55:42.589124 27332 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:55:42.589087 27301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.593954 27315 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37dc325715bd44f6bd9f1fabb76a2e4a. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.594233 27315 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.595907 27330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.596143 27330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:42.606575 27315 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } }
I20260806 07:55:42.606801 27315 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:42.628926 27332 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:55:42.631006 27332 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:55:42.668182 27332 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: Generated new TSK 0
I20260806 07:55:42.669567 27332 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:42.835539 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:43.095896 27312 catalog_manager.cc:1295] Loaded cluster ID: e3a8337d3f3d44678292dda3271750e8
I20260806 07:55:43.095988 27312 catalog_manager.cc:1588] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: loading cluster ID for follower catalog manager: success
I20260806 07:55:43.098619 27312 catalog_manager.cc:1610] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: acquiring CA information for follower catalog manager: success
I20260806 07:55:43.100478 27312 catalog_manager.cc:1638] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:55:43.184468 27345 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:43.184813 27345 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:43.184973 27345 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:43.193388 27345 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:43.193684 27345 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:43.195123 27326 catalog_manager.cc:1295] Loaded cluster ID: e3a8337d3f3d44678292dda3271750e8
I20260806 07:55:43.195199 27326 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: loading cluster ID for follower catalog manager: success
I20260806 07:55:43.197911 27326 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: acquiring CA information for follower catalog manager: success
I20260806 07:55:43.199709 27326 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:43.204479 27345 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:43.206045 27345 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:43.207368 27345 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:43.217003 27352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:43.217026 27351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:43.217844 27354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:43.219029 27345 server_base.cc:1061] running on GCE node
I20260806 07:55:43.219623 27345 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:43.219789 27345 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:43.219901 27345 hybrid_clock.cc:648] HybridClock initialized: now 1786002943219860 us; error 0 us; skew 500 ppm
I20260806 07:55:43.220223 27345 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:43.223568 27345 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:43.225194 27345 webserver.cc:533] Webserver started at http://127.20.202.193:42877/ using document root <none> and password file <none>
I20260806 07:55:43.226017 27345 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:43.226154 27345 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:43.226527 27345 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:43.228925 27345 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:43.229796 27345 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:43.235155 27345 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:55:43.238603 27360 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:43.240099 27345 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:43.240378 27345 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:43.240578 27345 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:43.261229 27345 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:43.265477 27345 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:43.265816 27345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:43.266899 27345 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:43.268652 27345 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:43.268759 27345 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:43.268926 27345 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:43.269006 27345 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:43.304939 27345 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33581
I20260806 07:55:43.305032 27475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33581 every 8 connection(s)
I20260806 07:55:43.306422 27345 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:43.307335 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27345
I20260806 07:55:43.307591 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:43.338171 27477 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46431
I20260806 07:55:43.338604 27477 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:43.339622 27477 heartbeater.cc:507] Master 127.20.202.254:46431 requested a full tablet report, sending...
I20260806 07:55:43.340521 27476 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36857
I20260806 07:55:43.340799 27476 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:43.341436 27476 heartbeater.cc:507] Master 127.20.202.252:36857 requested a full tablet report, sending...
I20260806 07:55:43.342471 27478 heartbeater.cc:344] Connected to a master server at 127.20.202.253:42153
I20260806 07:55:43.342417 27090 ts_manager.cc:194] Registered new tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:43.342603 27478 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:43.343052 27478 heartbeater.cc:507] Master 127.20.202.253:42153 requested a full tablet report, sending...
I20260806 07:55:43.343122 27243 ts_manager.cc:194] Registered new tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:43.344913 27166 ts_manager.cc:194] Registered new tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:43.345227 27243 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:52211
I20260806 07:55:43.350538 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:43.489269 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:36799
--webserver_interface=127.20.202.251
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:43.850107 27490 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:43.850466 27490 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:43.850657 27490 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:43.859084 27490 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:43.859179 27490 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:43.859274 27490 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:43.859352 27490 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:43.870373 27490 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:36799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:43.872118 27490 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:43.873468 27490 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:43.882638 27495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:43.882612 27496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:43.882858 27498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:43.884189 27490 server_base.cc:1061] running on GCE node
I20260806 07:55:43.884949 27490 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:43.885147 27490 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:43.885245 27490 hybrid_clock.cc:648] HybridClock initialized: now 1786002943885224 us; error 0 us; skew 500 ppm
I20260806 07:55:43.885637 27490 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:43.889187 27490 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:55:43.891204 27490 webserver.cc:533] Webserver started at http://127.20.202.251:44543/ using document root <none> and password file <none>
I20260806 07:55:43.891911 27490 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:43.892004 27490 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:43.892305 27490 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:43.894826 27490 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "8cb830a8e52c49059aac0ce0fd173768"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:43.895553 27490 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "8cb830a8e52c49059aac0ce0fd173768"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:43.901021 27490 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:55:43.904731 27504 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:43.906311 27490 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:43.906503 27490 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "8cb830a8e52c49059aac0ce0fd173768"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:43.906679 27490 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:43.932055 27490 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:43.936493 27490 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:43.936810 27490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:43.962225 27557 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:36799 every 8 connection(s)
I20260806 07:55:43.962255 27490 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:36799
I20260806 07:55:43.963817 27490 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:43.968659 27558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:43.972678 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27490
I20260806 07:55:43.976164 27558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:55:43 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:43.998616 27558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:44.013211 27558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:44.028286 27558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:55:44.045823 27558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Bootstrap starting.
I20260806 07:55:44.048949 27558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:44.050115 27558 log.cc:826] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:44.052891 27558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: No bootstrap required, opened a new log
I20260806 07:55:44.058043 27558 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } }
I20260806 07:55:44.058524 27558 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:44.058647 27558 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb830a8e52c49059aac0ce0fd173768, State: Initialized, Role: FOLLOWER
I20260806 07:55:44.059361 27558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } }
I20260806 07:55:44.060325 27567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } } }
I20260806 07:55:44.060693 27567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:44.061286 27558 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:44.072157 27579 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:44.072378 27579 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:44.081012 27490 master_runner.cc:429] Detected that this master 8cb830a8e52c49059aac0ce0fd173768 is joining an existing cluster
I20260806 07:55:44.081202 27490 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:55:44.082280 27490 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:55:44.082674 27490 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:55:44.082933 27490 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: stopping tablet replica
I20260806 07:55:44.084646 27490 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:36799
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002941, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:55:44.147179 27243 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:36799: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 36799
}
attrs {
promote: true
}
}
dest_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a"
cas_config_opid_index: -1
I20260806 07:55:44.149133 27243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } }
I20260806 07:55:44.152575 27100 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:55:44.153275 27228 consensus_peers.cc:597] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a -> Peer 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251:36799): Couldn't send request to peer 8cb830a8e52c49059aac0ce0fd173768. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:44.153299 27287 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:44.156610 27175 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 1 FOLLOWER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:55:44.157382 27287 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:44.158643 27592 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.159365 27100 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.163512 27592 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:36799
I20260806 07:55:44.164378 27490 master.cc:562] Master@127.20.202.251:36799 shutting down...
I20260806 07:55:44.185922 27175 raft_consensus.cc:2955] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.196079 27490 master.cc:584] Master@127.20.202.251:36799 shutdown complete.
I20260806 07:55:44.197103 27490 master_runner.cc:306] Clearing existing system tablet
I20260806 07:55:44.197862 27490 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:44.198653 27490 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:44.208829 27598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.209446 27598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:44.210314 27490 fs_manager.cc:714] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260806 07:55:44.211386 27597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.211808 27597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:44.212694 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.212661 27287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.212620 27596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.213083 27596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:44.213212 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:44.213056 27287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:44.214166 27601 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:44.214581 27490 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:44.214864 27490 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 07:55:44.214986 27490 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "8cb830a8e52c49059aac0ce0fd173768"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:44.218489 27490 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:55:44.223321 27490 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:55:44.223518 27490 log.cc:1199] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:55:44.223917 27490 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Deleting consensus metadata
I20260806 07:55:44.224179 27490 master_runner.cc:316] Copying system tablet from 127.20.202.252:36857
I20260806 07:55:44.226727 27490 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.252:36857
I20260806 07:55:44.241886 27263 tablet_copy_service.cc:140] P 37dc325715bd44f6bd9f1fabb76a2e4a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8cb830a8e52c49059aac0ce0fd173768 ({username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:55720)
I20260806 07:55:44.242252 27263 tablet_copy_service.cc:161] P 37dc325715bd44f6bd9f1fabb76a2e4a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8cb830a8e52c49059aac0ce0fd173768 at {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:55720: session id = 8cb830a8e52c49059aac0ce0fd173768-00000000000000000000000000000000
I20260806 07:55:44.247126 27263 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:55:44.249706 27490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:55:44.254482 27490 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: tablet copy: Starting download of 0 data blocks...
I20260806 07:55:44.254801 27490 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: tablet copy: Starting download of 1 WAL segments...
I20260806 07:55:44.258351 27490 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:55:44.263434 27263 tablet_copy_service.cc:342] P 37dc325715bd44f6bd9f1fabb76a2e4a: Request end of tablet copy session 8cb830a8e52c49059aac0ce0fd173768-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:55720
I20260806 07:55:44.263715 27263 tablet_copy_service.cc:434] P 37dc325715bd44f6bd9f1fabb76a2e4a: ending tablet copy session 8cb830a8e52c49059aac0ce0fd173768-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8cb830a8e52c49059aac0ce0fd173768
I20260806 07:55:44.275473 27490 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:44.278534 27609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:44.278653 27610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:44.279357 27612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:44.280268 27490 server_base.cc:1061] running on GCE node
I20260806 07:55:44.280575 27490 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:44.280665 27490 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:44.280807 27490 hybrid_clock.cc:648] HybridClock initialized: now 1786002944280747 us; error 0 us; skew 500 ppm
I20260806 07:55:44.281009 27490 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:44.283123 27490 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:55:44.284327 27490 webserver.cc:533] Webserver started at http://127.20.202.251:34327/ using document root <none> and password file <none>
I20260806 07:55:44.284718 27490 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:44.284785 27490 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:44.287456 27490 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 07:55:44.289240 27618 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:44.289721 27490 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 07:55:44.289858 27490 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "8cb830a8e52c49059aac0ce0fd173768"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:44.290009 27490 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:44.348335 27476 heartbeater.cc:499] Master 127.20.202.252:36857 was elected leader, sending a full tablet report...
I20260806 07:55:44.349751 27490 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:44.353713 27490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:44.371934 27490 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:36799
I20260806 07:55:44.371968 27670 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:36799 every 8 connection(s)
I20260806 07:55:44.373127 27490 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:44.376801 27671 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:44.379418 27671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Bootstrap starting.
I20260806 07:55:44.392769 27671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:44.393339 27671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: Bootstrap complete.
I20260806 07:55:44.394323 27671 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } }
I20260806 07:55:44.394526 27671 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8cb830a8e52c49059aac0ce0fd173768, State: Initialized, Role: LEARNER
I20260806 07:55:44.394786 27671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } }
I20260806 07:55:44.395612 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.395937 27671 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:44.395943 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:44.404922 27685 catalog_manager.cc:1295] Loaded cluster ID: e3a8337d3f3d44678292dda3271750e8
I20260806 07:55:44.405050 27685 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: loading cluster ID for follower catalog manager: success
I20260806 07:55:44.407881 27685 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: acquiring CA information for follower catalog manager: success
I20260806 07:55:44.410059 27685 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:44.429302 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27345
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:55:44.448642 27471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:44.538932 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27345
I20260806 07:55:44.559182 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33581
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--webserver_port=42877
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:44.632200 27646 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:55:44.633033 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37dc325715bd44f6bd9f1fabb76a2e4a. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:44.633303 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:55:44.920547 27689 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:44.920869 27689 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:44.920969 27689 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:44.929488 27689 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:44.929798 27689 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:44.943256 27689 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42877
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:44.945405 27689 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:44.947184 27689 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:44.958691 27694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:44.958719 27695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:44.960508 27697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:44.960914 27689 server_base.cc:1061] running on GCE node
I20260806 07:55:44.961618 27689 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:44.961805 27689 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:44.961894 27689 hybrid_clock.cc:648] HybridClock initialized: now 1786002944961860 us; error 0 us; skew 500 ppm
I20260806 07:55:44.962213 27689 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:44 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:44.965516 27689 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:44.967149 27689 webserver.cc:533] Webserver started at http://127.20.202.193:42877/ using document root <none> and password file <none>
I20260806 07:55:44.967905 27689 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:44.968057 27689 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:44.973729 27689 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:55:44.976691 27703 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:44.978408 27689 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:44.978612 27689 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:44.979414 27689 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:45.012334 27689 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:45.017449 27689 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:45.017871 27689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:45.019114 27689 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:45.021091 27689 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:45.021242 27689 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:45.021401 27689 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:45.021513 27689 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:45.041852 27596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: true } } }
I20260806 07:55:45.042294 27596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:45.045179 27762 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a: attempting to promote NON_VOTER 8cb830a8e52c49059aac0ce0fd173768 to VOTER
I20260806 07:55:45.046756 27762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } }
I20260806 07:55:45.049870 27646 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 1 LEARNER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:45.050328 27175 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 1 FOLLOWER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:45.050745 27762 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:45.050951 27100 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Refusing update from remote peer 37dc325715bd44f6bd9f1fabb76a2e4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:45.051348 27759 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:45.051792 27596 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:45.055850 27759 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.057734 27175 raft_consensus.cc:2955] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.057866 27646 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.062323 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.062626 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb830a8e52c49059aac0ce0fd173768 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:45.063484 27100 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8cb830a8e52c49059aac0ce0fd173768 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.065959 27759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.066471 27596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.066993 27596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:45.067238 27596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.067481 27596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:45.067749 27759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:45.069360 27784 sys_catalog.cc:455] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.069727 27784 sys_catalog.cc:458] T 00000000000000000000000000000000 P e555fb35a6544a1999855b330e894575 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:45.069845 27778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.070232 27778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:45.071273 27788 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.071646 27788 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:45.086158 27689 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33581
I20260806 07:55:45.086248 27826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33581 every 8 connection(s)
I20260806 07:55:45.088006 27689 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:45.094730 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27689
Aug 06 07:55:45 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:55:45 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:45 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:55:45 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:45.125399 27827 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36799
I20260806 07:55:45.125910 27827 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:45.126081 27830 heartbeater.cc:344] Connected to a master server at 127.20.202.253:42153
I20260806 07:55:45.126253 27830 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:45.126974 27827 heartbeater.cc:507] Master 127.20.202.251:36799 requested a full tablet report, sending...
I20260806 07:55:45.127894 27829 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46431
I20260806 07:55:45.128058 27829 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:45.128145 27830 heartbeater.cc:507] Master 127.20.202.253:42153 requested a full tablet report, sending...
I20260806 07:55:45.128556 27829 heartbeater.cc:507] Master 127.20.202.254:46431 requested a full tablet report, sending...
I20260806 07:55:45.129650 27166 ts_manager.cc:194] Re-registered known tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:45.129874 27090 ts_manager.cc:194] Re-registered known tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:45.130702 27828 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36857
I20260806 07:55:45.130864 27828 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:45.131268 27828 heartbeater.cc:507] Master 127.20.202.252:36857 requested a full tablet report, sending...
I20260806 07:55:45.131489 27636 ts_manager.cc:194] Registered new tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:45.132385 27243 ts_manager.cc:194] Re-registered known tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:45.133204 27243 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:33023
I20260806 07:55:45.138204 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:45.146749 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:45.146935 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27058
I20260806 07:55:45.147886 27120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:45.258613 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27058
I20260806 07:55:45.283463 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46431
--webserver_interface=127.20.202.254
--webserver_port=37983
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:45.480547 27230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:46431: connect: Connection refused (error 111)
W20260806 07:55:45.482909 27230 consensus_peers.cc:597] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a -> Peer f830aa49ae2c4b3287ff111892b706d6 (127.20.202.254:46431): Couldn't send request to peer f830aa49ae2c4b3287ff111892b706d6. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:46431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:55:45.669157 27840 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:45.669545 27840 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:45.669709 27840 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:45.678591 27840 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:45.678723 27840 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:45.678833 27840 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:45.678912 27840 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:45.691222 27840 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:46431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=37983
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:45.693015 27840 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:45.694617 27840 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:45.702431 27849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:45.703709 27852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:45.704085 27850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:45.704846 27840 server_base.cc:1061] running on GCE node
I20260806 07:55:45.705703 27840 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:45.705904 27840 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:45.706005 27840 hybrid_clock.cc:648] HybridClock initialized: now 1786002945705982 us; error 0 us; skew 500 ppm
I20260806 07:55:45.706359 27840 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:45 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002945, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:45.710139 27840 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:55:45.712317 27840 webserver.cc:533] Webserver started at http://127.20.202.254:37983/ using document root <none> and password file <none>
I20260806 07:55:45.713089 27840 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:45.713238 27840 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:45.718958 27840 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:55:45.721921 27858 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:45.723292 27840 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:45.723470 27840 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "f830aa49ae2c4b3287ff111892b706d6"
format_stamp: "Formatted at 2026-08-06 07:55:40 on dist-test-slave-g350"
I20260806 07:55:45.724220 27840 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:45.742624 27840 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:45.746999 27840 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:45.747314 27840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:45.769418 27840 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:46431
I20260806 07:55:45.769440 27910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:46431 every 8 connection(s)
I20260806 07:55:45.771186 27840 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:45.776049 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27840
I20260806 07:55:45.776381 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27689
I20260806 07:55:45.780035 27911 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:45.784713 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Bootstrap starting.
I20260806 07:55:45.795166 27822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:45.806022 27911 log.cc:826] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:45.816344 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:45.816946 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: Bootstrap complete.
I20260806 07:55:45.824069 27911 raft_consensus.cc:359] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } }
I20260806 07:55:45.824841 27911 raft_consensus.cc:740] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f830aa49ae2c4b3287ff111892b706d6, State: Initialized, Role: FOLLOWER
I20260806 07:55:45.825668 27911 consensus_queue.cc:260] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } }
I20260806 07:55:45.826592 27917 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:45.826930 27917 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:45.827800 27911 sys_catalog.cc:565] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:45.839103 27928 catalog_manager.cc:1295] Loaded cluster ID: e3a8337d3f3d44678292dda3271750e8
I20260806 07:55:45.839325 27928 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: loading cluster ID for follower catalog manager: success
I20260806 07:55:45.844764 27928 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: acquiring CA information for follower catalog manager: success
I20260806 07:55:45.848147 27928 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:45.912751 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27689
I20260806 07:55:45.932644 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33581
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--webserver_port=42877
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:46.013227 27778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:46.013509 27778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:46.024853 27917 sys_catalog.cc:455] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37dc325715bd44f6bd9f1fabb76a2e4a. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:46.025132 27917 sys_catalog.cc:458] T 00000000000000000000000000000000 P f830aa49ae2c4b3287ff111892b706d6 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:55:46.313956 27935 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:46.314280 27935 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:46.314477 27935 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:46.324007 27935 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:46.324287 27935 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:46.340652 27935 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42877
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:46.342592 27935 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:46.344298 27935 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:46.355636 27945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:46.355733 27944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:46.355760 27947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:46.357331 27935 server_base.cc:1061] running on GCE node
I20260806 07:55:46.358079 27935 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:46.358270 27935 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:46.358382 27935 hybrid_clock.cc:648] HybridClock initialized: now 1786002946358341 us; error 0 us; skew 500 ppm
I20260806 07:55:46.358721 27935 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:46 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:46.362226 27935 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:46.363982 27935 webserver.cc:533] Webserver started at http://127.20.202.193:42877/ using document root <none> and password file <none>
I20260806 07:55:46.364746 27935 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:46.364887 27935 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:46.370667 27935 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:55:46.373970 27953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:46.375612 27935 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:55:46.375801 27935 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:46.376608 27935 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:46.406113 27935 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:46.410799 27935 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:46.411152 27935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:46.412209 27935 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:46.413856 27935 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:46.413945 27935 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:46.414036 27935 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:46.414091 27935 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:46.451018 27935 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33581
I20260806 07:55:46.451170 28068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33581 every 8 connection(s)
I20260806 07:55:46.452682 27935 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:46.463052 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 27935
Aug 06 07:55:46 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:55:46 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:55:46 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:55:46 dist-test-slave-g350 krb5kdc[27044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:46.521154 28069 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36799
I20260806 07:55:46.521656 28069 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:46.522886 28069 heartbeater.cc:507] Master 127.20.202.251:36799 requested a full tablet report, sending...
I20260806 07:55:46.524240 28070 heartbeater.cc:344] Connected to a master server at 127.20.202.252:36857
I20260806 07:55:46.524437 28070 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:46.525076 28070 heartbeater.cc:507] Master 127.20.202.252:36857 requested a full tablet report, sending...
I20260806 07:55:46.525367 27636 ts_manager.cc:194] Re-registered known tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:46.526438 27243 ts_manager.cc:194] Re-registered known tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:46.527712 27243 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:57323
I20260806 07:55:46.529894 28072 heartbeater.cc:344] Connected to a master server at 127.20.202.253:42153
I20260806 07:55:46.530087 28072 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:46.530478 28072 heartbeater.cc:507] Master 127.20.202.253:42153 requested a full tablet report, sending...
I20260806 07:55:46.531141 28071 heartbeater.cc:344] Connected to a master server at 127.20.202.254:46431
I20260806 07:55:46.531322 28071 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:46.531602 28071 heartbeater.cc:507] Master 127.20.202.254:46431 requested a full tablet report, sending...
I20260806 07:55:46.531898 27166 ts_manager.cc:194] Re-registered known tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:46.533802 27875 ts_manager.cc:194] Registered new tserver with Master: 03208f4b6d964f89b4a4122f0108fbd6 (127.20.202.193:33581)
I20260806 07:55:46.559888 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27935
W20260806 07:55:46.561183 28084 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:46.557125 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:45386 (local address 127.20.202.193:33581)
0806 07:55:46.557874 (+ 749us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:46.557883 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:46.557921 (+ 38us) server_negotiation.cc:408] Connection header received
0806 07:55:46.558103 (+ 182us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:46.558112 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:46.558338 (+ 226us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:46.558544 (+ 206us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:46.559641 (+ 1097us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45386: BlockingRecv error: recv got EOF from 127.0.0.1:45386 (error 108)
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":339,"threads_started":1}
W20260806 07:55:46.562526 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27935: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33581: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:55:46.562673 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27935
I20260806 07:55:46.582075 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33581
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--webserver_port=42877
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:46.628297 27937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f830aa49ae2c4b3287ff111892b706d6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 46431 } } peers { permanent_uuid: "e555fb35a6544a1999855b330e894575" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 42153 } } peers { permanent_uuid: "37dc325715bd44f6bd9f1fabb76a2e4a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 36857 } } peers { permanent_uuid: "8cb830a8e52c49059aac0ce0fd173768" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36799 } attrs { promote: false } } }
I20260806 07:55:46.628659 27937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a [sys.catalog]: This master's current role is: LEADER
W20260806 07:55:46.948168 28086 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:46.948553 28086 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:46.948719 28086 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:55:46.948810 28086 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:46.957533 28086 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:46.957828 28086 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:46.968755 28086 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42877
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:46.970849 28086 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:46.972287 28086 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:46.982596 28094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:46.982904 28097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:46.983276 28095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:46.983835 28086 server_base.cc:1061] running on GCE node
I20260806 07:55:46.984470 28086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:46.984655 28086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:46.984776 28086 hybrid_clock.cc:648] HybridClock initialized: now 1786002946984733 us; error 0 us; skew 500 ppm
I20260806 07:55:46.985093 28086 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:46 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:46.988761 28086 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:46.990664 28086 webserver.cc:533] Webserver started at http://127.20.202.193:42877/ using document root <none> and password file <none>
I20260806 07:55:46.991506 28086 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:46.991628 28086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:46.997480 28086 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:55:47.000543 28103 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:47.002379 28086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:55:47.002578 28086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:47.003404 28086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:47.025440 28086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:47.033229 28086 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:47.033658 28086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:47.034910 28086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:47.036818 28086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:47.036947 28086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:47.037109 28086 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:47.037187 28086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:55:47.037732 28086 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:55:47.037828 28086 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:55:47.037935 28086 fault_injection.cc:59] injecting fault for kudu (pid 28086): FLAGS_fault_before_start (process will exit)
I20260806 07:55:47.056293 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:33581
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857
--webserver_port=42877
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:47.422250 28111 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:47.422614 28111 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:47.422786 28111 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:47.431452 28111 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:47.431753 28111 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:47.442889 28111 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:33581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42877
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:46431,127.20.202.253:42153,127.20.202.252:36857,127.20.202.251:36799
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:47.444743 28111 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:47.446475 28111 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:47.457605 28119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:47.457811 28118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:47.459081 28121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:47.459246 28111 server_base.cc:1061] running on GCE node
I20260806 07:55:47.459884 28111 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:55:47.460089 28111 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:55:47.460187 28111 hybrid_clock.cc:648] HybridClock initialized: now 1786002947460165 us; error 0 us; skew 500 ppm
I20260806 07:55:47.460512 28111 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:47 dist-test-slave-g350 krb5kdc[27044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002947, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:47.463948 28111 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:47.465700 28111 webserver.cc:533] Webserver started at http://127.20.202.193:42877/ using document root <none> and password file <none>
I20260806 07:55:47.466419 28111 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:47.466517 28111 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:47.471983 28111 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:55:47.474886 28127 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:47.476425 28111 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:47.476593 28111 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "03208f4b6d964f89b4a4122f0108fbd6"
format_stamp: "Formatted at 2026-08-06 07:55:43 on dist-test-slave-g350"
I20260806 07:55:47.477353 28111 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:47.494860 28111 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:47.499110 28111 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:47.499400 28111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:47.500521 28111 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:47.502279 28111 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:47.502370 28111 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:47.502506 28111 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:47.502589 28111 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:47.539011 28111 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33581
I20260806 07:55:47.539140 28241 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33581 every 8 connection(s)
I20260806 07:55:47.540589 28111 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:47.541731 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28111
I20260806 07:55:47.542196 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28111
W20260806 07:55:47.576447 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28111: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:33581: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:55:47.576666 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28111
I20260806 07:55:47.596983 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27840
W20260806 07:55:47.597195 28113 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:47.552939 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:45127 (local address 127.20.202.254:46431)
0806 07:55:47.553205 (+ 266us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:47.553216 (+ 11us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:47.564819 (+ 11603us) server_negotiation.cc:408] Connection header received
0806 07:55:47.573432 (+ 8613us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:47.573440 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:47.573541 (+ 101us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:47.573660 (+ 119us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:47.596246 (+ 22586us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:45127: BlockingRecv error: recv got EOF from 127.20.202.193:45127 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260806 07:55:47.597208 28253 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:47.560886 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:46245 (local address 127.20.202.253:42153)
0806 07:55:47.569984 (+ 9098us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:47.569989 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:47.570313 (+ 324us) server_negotiation.cc:408] Connection header received
0806 07:55:47.573535 (+ 3222us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:47.573541 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:47.573674 (+ 133us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:47.573785 (+ 111us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:47.596291 (+ 22506us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:46245: BlockingRecv error: recv got EOF from 127.20.202.193:46245 (error 108)
Metrics: {"server-negotiator.queue_time_us":8998,"thread_start_us":8870,"threads_started":1}
W20260806 07:55:47.597215 28255 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:47.574834 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:57505 (local address 127.20.202.251:36799)
0806 07:55:47.579465 (+ 4631us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:47.579469 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:47.596206 (+ 16737us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:57505: BlockingRecv error: recv got EOF from 127.20.202.193:57505 (error 108)
Metrics: {"server-negotiator.queue_time_us":4538,"thread_start_us":4448,"threads_started":1}
W20260806 07:55:47.598802 28249 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:47.546469 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:57099 (local address 127.20.202.252:36857)
0806 07:55:47.558863 (+ 12394us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:47.558874 (+ 11us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:47.570676 (+ 11802us) server_negotiation.cc:408] Connection header received
0806 07:55:47.573477 (+ 2801us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:47.573489 (+ 12us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:47.573651 (+ 162us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:47.573775 (+ 124us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:47.597846 (+ 24071us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:57099: BlockingRecv error: recv got EOF from 127.20.202.193:57099 (error 108)
Metrics: {"server-negotiator.queue_time_us":12212,"thread_start_us":12031,"threads_started":1}
W20260806 07:55:47.599697 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27840: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:46431: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:55:47.599850 28113 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:47.598104 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.20.202.254:46431)
0806 07:55:47.598304 (+ 200us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:47.598310 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:47.598333 (+ 23us) server_negotiation.cc:408] Connection header received
0806 07:55:47.598430 (+ 97us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:47.598436 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:47.598608 (+ 172us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:47.598743 (+ 135us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:47.599627 (+ 884us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48074: BlockingRecv error: recv got EOF from 127.0.0.1:48074 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 07:55:47.599884 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27840
I20260806 07:55:47.624178 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27135
I20260806 07:55:47.625476 27196 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:55:47.642908 27230 consensus_peers.cc:597] T 00000000000000000000000000000000 P 37dc325715bd44f6bd9f1fabb76a2e4a -> Peer f830aa49ae2c4b3287ff111892b706d6 (127.20.202.254:46431): Couldn't send request to peer f830aa49ae2c4b3287ff111892b706d6. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:46431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:47.743925 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27135
I20260806 07:55:47.767976 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27212
I20260806 07:55:47.769268 27273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:47.893117 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27212
W20260806 07:55:47.927054 27619 connection.cc:573] server connection from 127.0.0.1:59978 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:55:47.927464 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 27490
I20260806 07:55:47.928822 27666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:48.050400 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 27490
[ OK ] ExternalMiniClusterTest.TestAddMaster/6 (10267 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/7
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:55:48 dist-test-slave-g350 krb5kdc[28264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:55:48 dist-test-slave-g350 krb5kdc[28264](info): set up 2 sockets
Aug 06 07:55:48 dist-test-slave-g350 krb5kdc[28264](info): commencing operation
krb5kdc: starting...
W20260806 07:55:50.169329 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.080s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:55:50 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:55:50Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:55:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:50.560510 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35425
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:50.919950 28281 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:50.920308 28281 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:50.920506 28281 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:50.929001 28281 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:50.929124 28281 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:50.929226 28281 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:50.929313 28281 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:50.940325 28281 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:50.942104 28281 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:50.943572 28281 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:50.952256 28286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:50.952227 28287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:50.953118 28289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:50.953608 28281 server_base.cc:1061] running on GCE node
I20260806 07:55:50.954382 28281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:50.955859 28281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:50.957084 28281 hybrid_clock.cc:648] HybridClock initialized: now 1786002950957002 us; error 52 us; skew 500 ppm
I20260806 07:55:50.957454 28281 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:50 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:50.960690 28281 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:55:50.962471 28281 webserver.cc:533] Webserver started at http://127.20.202.254:38037/ using document root <none> and password file <none>
I20260806 07:55:50.963239 28281 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:50.963374 28281 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:50.963734 28281 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:50.966334 28281 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "b74cd3d7a919435d9fb276f1db033d08"
format_stamp: "Formatted at 2026-08-06 07:55:50 on dist-test-slave-g350"
I20260806 07:55:50.967162 28281 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "b74cd3d7a919435d9fb276f1db033d08"
format_stamp: "Formatted at 2026-08-06 07:55:50 on dist-test-slave-g350"
I20260806 07:55:50.973008 28281 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 07:55:50.976686 28296 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:50.978226 28281 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:55:50.978412 28281 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "b74cd3d7a919435d9fb276f1db033d08"
format_stamp: "Formatted at 2026-08-06 07:55:50 on dist-test-slave-g350"
I20260806 07:55:50.978619 28281 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:51.016953 28281 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:51.021181 28281 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:51.021523 28281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:51.042920 28281 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:35425
I20260806 07:55:51.043100 28348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:35425 every 8 connection(s)
I20260806 07:55:51.044517 28281 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:51.049153 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:51.053505 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28281
I20260806 07:55:51.053768 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:55:51.057282 28349 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:51 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:51.084327 28349 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:51.085819 28299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45847: connect: Connection refused (error 111)
W20260806 07:55:51.087672 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45847: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45847: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:51.137089 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } attempt: 1
W20260806 07:55:51.139636 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45847: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45847: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:51.197126 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45847
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:51.207924 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } attempt: 2
W20260806 07:55:51.210325 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45847: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45847: connect: Connection refused (error 111)
I20260806 07:55:51.301621 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } attempt: 3
W20260806 07:55:51.303845 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45847: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45847: connect: Connection refused (error 111)
I20260806 07:55:51.447160 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } attempt: 4
W20260806 07:55:51.449653 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:45847: Network error: Client connection negotiation failed: client connection to 127.20.202.253:45847: connect: Connection refused (error 111)
W20260806 07:55:51.561282 28358 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:51.561666 28358 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:51.561818 28358 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:51.570314 28358 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:51.570413 28358 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:51.570516 28358 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:51.570585 28358 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:51.581480 28358 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:51.583246 28358 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:51.584695 28358 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:51.593214 28363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:51.593220 28364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:51.594254 28366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:51.595090 28358 server_base.cc:1061] running on GCE node
I20260806 07:55:51.595981 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:51.597462 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:51.598704 28358 hybrid_clock.cc:648] HybridClock initialized: now 1786002951598662 us; error 56 us; skew 500 ppm
I20260806 07:55:51.599071 28358 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:51 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002951, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:51.602315 28358 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:55:51.604061 28358 webserver.cc:533] Webserver started at http://127.20.202.253:40015/ using document root <none> and password file <none>
I20260806 07:55:51.604846 28358 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:51.604980 28358 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:51.605366 28358 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:51.607971 28358 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "bff9b89dfc944ce5b093eae9b10075fe"
format_stamp: "Formatted at 2026-08-06 07:55:51 on dist-test-slave-g350"
I20260806 07:55:51.608973 28358 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "bff9b89dfc944ce5b093eae9b10075fe"
format_stamp: "Formatted at 2026-08-06 07:55:51 on dist-test-slave-g350"
I20260806 07:55:51.614760 28358 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:55:51.618264 28373 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:51.619689 28358 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:51.619865 28358 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "bff9b89dfc944ce5b093eae9b10075fe"
format_stamp: "Formatted at 2026-08-06 07:55:51 on dist-test-slave-g350"
I20260806 07:55:51.620074 28358 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:51.636544 28358 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:51.640630 28358 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:51.640920 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:51.662796 28358 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:45847
I20260806 07:55:51.662818 28425 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:45847 every 8 connection(s)
I20260806 07:55:51.664453 28358 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:55:51.669217 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:51.670593 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28358
I20260806 07:55:51.670804 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:55:51.675408 28426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:51 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002951, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:51.699203 28426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:55:51 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002951, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:51.716297 28426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:51.717433 28376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
W20260806 07:55:51.719203 28426 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:51.748979 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } attempt: 5
Aug 06 07:55:51 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:51.764945 28349 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:55:51.766878 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
I20260806 07:55:51.768530 28426 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 1
W20260806 07:55:51.770459 28426 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:51.818120 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:40393
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:51.818145 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 1
W20260806 07:55:51.820389 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
I20260806 07:55:51.838714 28426 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 2
W20260806 07:55:51.841151 28426 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
I20260806 07:55:51.888676 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 2
W20260806 07:55:51.890877 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
I20260806 07:55:51.932407 28426 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 3
W20260806 07:55:51.934569 28426 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
I20260806 07:55:51.997154 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 3
W20260806 07:55:51.999572 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
I20260806 07:55:52.077814 28426 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 4
W20260806 07:55:52.079939 28426 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
W20260806 07:55:52.175616 28436 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:52.175992 28436 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:52.176148 28436 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:55:52.176841 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 4
W20260806 07:55:52.179173 28349 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40393: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40393: connect: Connection refused (error 111)
W20260806 07:55:52.185861 28436 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:52.185952 28436 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:52.186017 28436 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:52.186066 28436 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:52.196918 28436 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:52.198601 28436 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:52.199945 28436 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:52.208195 28441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:52.208372 28442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:52.209334 28444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:52.209924 28436 server_base.cc:1061] running on GCE node
I20260806 07:55:52.210659 28436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:52.212040 28436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:52.213277 28436 hybrid_clock.cc:648] HybridClock initialized: now 1786002952213223 us; error 47 us; skew 500 ppm
I20260806 07:55:52.213690 28436 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:52.216872 28436 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:55:52.218535 28436 webserver.cc:533] Webserver started at http://127.20.202.252:43199/ using document root <none> and password file <none>
I20260806 07:55:52.219254 28436 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:52.219383 28436 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:52.219717 28436 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:52.222201 28436 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "5a612facfe31495c9fc2d20376ca161a"
format_stamp: "Formatted at 2026-08-06 07:55:52 on dist-test-slave-g350"
I20260806 07:55:52.222966 28436 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "5a612facfe31495c9fc2d20376ca161a"
format_stamp: "Formatted at 2026-08-06 07:55:52 on dist-test-slave-g350"
I20260806 07:55:52.228560 28436 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:55:52.232156 28451 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:52.233558 28436 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:55:52.233796 28436 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "5a612facfe31495c9fc2d20376ca161a"
format_stamp: "Formatted at 2026-08-06 07:55:52 on dist-test-slave-g350"
I20260806 07:55:52.234012 28436 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:52.247714 28436 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:52.258402 28436 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:52.258762 28436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:52.282205 28436 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:40393
I20260806 07:55:52.282228 28502 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:40393 every 8 connection(s)
I20260806 07:55:52.284008 28436 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:55:52.288442 28503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:52.291523 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28436
I20260806 07:55:52.291733 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:55:52.296540 28503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:52.329563 28503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:52.346465 28503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:52.366565 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: Bootstrap starting.
I20260806 07:55:52.370153 28503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:52.371273 28503 log.cc:826] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:52.374060 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: No bootstrap required, opened a new log
I20260806 07:55:52.379218 28426 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 5
I20260806 07:55:52.380045 28503 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.380511 28503 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:52.380713 28503 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a612facfe31495c9fc2d20376ca161a, State: Initialized, Role: FOLLOWER
I20260806 07:55:52.381466 28503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.382578 28515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.382905 28515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.383627 28503 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002951, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:52.392526 28436 master_runner.cc:187] Error getting master registration for 127.20.202.254:35425: OK, instance_id { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" instance_seqno: 1786002951037124 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:52.398455 28436 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:52.403264 28426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: Bootstrap starting.
W20260806 07:55:52.403806 28526 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:52.403941 28526 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:52.407595 28426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:52.408973 28426 log.cc:826] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:52.412034 28426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: No bootstrap required, opened a new log
I20260806 07:55:52.419924 28426 raft_consensus.cc:359] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.420310 28426 raft_consensus.cc:385] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:52.420463 28426 raft_consensus.cc:740] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff9b89dfc944ce5b093eae9b10075fe, State: Initialized, Role: FOLLOWER
I20260806 07:55:52.421288 28426 consensus_queue.cc:260] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.422318 28529 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.422606 28529 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.423781 28426 sys_catalog.cc:565] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:52.436223 28541 catalog_manager.cc:1594] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:52.436474 28541 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:52.433192 28358 master_runner.cc:187] Error getting master registration for 127.20.202.254:35425: OK, instance_id { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" instance_seqno: 1786002951037124 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:55:52.440738 28358 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:55:52.456460 28349 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } attempt: 5
Aug 06 07:55:52 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:52.480480 28349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Bootstrap starting.
I20260806 07:55:52.484948 28349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:52.486446 28349 log.cc:826] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:52.493223 28349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: No bootstrap required, opened a new log
I20260806 07:55:52.501235 28349 raft_consensus.cc:359] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.501652 28349 raft_consensus.cc:385] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:52.501799 28349 raft_consensus.cc:740] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74cd3d7a919435d9fb276f1db033d08, State: Initialized, Role: FOLLOWER
I20260806 07:55:52.502612 28349 consensus_queue.cc:260] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.503898 28544 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.504333 28544 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.505256 28349 sys_catalog.cc:565] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:52.511319 28457 tablet.cc:2406] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:55:52.525044 28302 tablet.cc:2406] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:55:52.532155 28555 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:52.532423 28555 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:52.535406 28281 master_runner.cc:264] No leader master found from master b74cd3d7a919435d9fb276f1db033d08
I20260806 07:55:52.535566 28281 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:55:52.649046 28379 tablet.cc:2406] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:55:52.709697 28515 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:55:52.710027 28515 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.711906 28515 leader_election.cc:290] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b74cd3d7a919435d9fb276f1db033d08 (127.20.202.254:35425), bff9b89dfc944ce5b093eae9b10075fe (127.20.202.253:45847)
I20260806 07:55:52.713790 28323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a612facfe31495c9fc2d20376ca161a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74cd3d7a919435d9fb276f1db033d08" is_pre_election: true
I20260806 07:55:52.714355 28323 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a612facfe31495c9fc2d20376ca161a in term 0.
I20260806 07:55:52.715063 28400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a612facfe31495c9fc2d20376ca161a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff9b89dfc944ce5b093eae9b10075fe" is_pre_election: true
I20260806 07:55:52.715515 28400 raft_consensus.cc:2468] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a612facfe31495c9fc2d20376ca161a in term 0.
I20260806 07:55:52.716701 28454 leader_election.cc:304] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a612facfe31495c9fc2d20376ca161a, bff9b89dfc944ce5b093eae9b10075fe; no voters:
I20260806 07:55:52.718495 28515 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:55:52.718665 28515 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:55:52.718770 28515 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:52.722196 28515 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.723446 28323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a612facfe31495c9fc2d20376ca161a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74cd3d7a919435d9fb276f1db033d08"
I20260806 07:55:52.723675 28323 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:52.727463 28323 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a612facfe31495c9fc2d20376ca161a in term 1.
I20260806 07:55:52.728153 28452 leader_election.cc:304] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a612facfe31495c9fc2d20376ca161a, b74cd3d7a919435d9fb276f1db033d08; no voters:
I20260806 07:55:52.732718 28400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a612facfe31495c9fc2d20376ca161a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff9b89dfc944ce5b093eae9b10075fe"
I20260806 07:55:52.732967 28400 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:55:52.736711 28400 raft_consensus.cc:2468] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a612facfe31495c9fc2d20376ca161a in term 1.
I20260806 07:55:52.737689 28515 leader_election.cc:290] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [CANDIDATE]: Term 1 election: Requested vote from peers b74cd3d7a919435d9fb276f1db033d08 (127.20.202.254:35425), bff9b89dfc944ce5b093eae9b10075fe (127.20.202.253:45847)
I20260806 07:55:52.738188 28515 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:55:52.738675 28515 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 1 LEADER]: Becoming Leader. State: Replica: 5a612facfe31495c9fc2d20376ca161a, State: Running, Role: LEADER
I20260806 07:55:52.739959 28515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } }
I20260806 07:55:52.759809 28557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a612facfe31495c9fc2d20376ca161a. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.760330 28557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:52.767432 28559 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:55:52.770249 28559 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:55:52.792016 28400 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 1 FOLLOWER]: Refusing update from remote peer 5a612facfe31495c9fc2d20376ca161a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:52.793480 28557 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:52.801623 28323 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Refusing update from remote peer 5a612facfe31495c9fc2d20376ca161a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:55:52.802850 28557 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:55:52.844677 28544 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a612facfe31495c9fc2d20376ca161a. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.845048 28544 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.847271 28560 mvcc.cc:204] Tried to move back new op lower bound from 7315468094620479488 to 7315468094451671040. Current Snapshot: MvccSnapshot[applied={T|T < 7315468094620479488}]
I20260806 07:55:52.850378 28563 mvcc.cc:204] Tried to move back new op lower bound from 7315468094620479488 to 7315468094451671040. Current Snapshot: MvccSnapshot[applied={T|T < 7315468094620479488}]
I20260806 07:55:52.859740 28515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.860064 28515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:52.861723 28559 catalog_manager.cc:1383] Generated new cluster ID: c38fcac4dc1b413b83f9cf05454e9f3b
I20260806 07:55:52.861848 28559 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:55:52.861886 28544 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.862178 28544 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.877460 28529 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a612facfe31495c9fc2d20376ca161a. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.877808 28529 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.882149 28557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.882400 28557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:52.884958 28562 mvcc.cc:204] Tried to move back new op lower bound from 7315468094620479488 to 7315468094451671040. Current Snapshot: MvccSnapshot[applied={T|T < 7315468094620479488}]
I20260806 07:55:52.895814 28529 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } }
I20260806 07:55:52.896072 28529 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:52.893581 28559 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:55:52.910930 28559 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:55:52.956645 28559 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: Generated new TSK 0
I20260806 07:55:52.958236 28559 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:53.157794 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:53.439188 28541 catalog_manager.cc:1295] Loaded cluster ID: c38fcac4dc1b413b83f9cf05454e9f3b
I20260806 07:55:53.439302 28541 catalog_manager.cc:1588] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: loading cluster ID for follower catalog manager: success
I20260806 07:55:53.443315 28541 catalog_manager.cc:1610] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: acquiring CA information for follower catalog manager: success
I20260806 07:55:53.446015 28541 catalog_manager.cc:1638] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:55:53.516073 28574 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:53.516383 28574 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:53.516474 28574 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:53.525027 28574 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:53.525269 28574 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:53.534540 28555 catalog_manager.cc:1295] Loaded cluster ID: c38fcac4dc1b413b83f9cf05454e9f3b
I20260806 07:55:53.534680 28555 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: loading cluster ID for follower catalog manager: success
I20260806 07:55:53.536590 28574 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:53.537691 28555 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: acquiring CA information for follower catalog manager: success
I20260806 07:55:53.538561 28574 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:53.539661 28555 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:53.540019 28574 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:53.550719 28581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:53.550736 28580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:53.551551 28583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:53.552286 28574 server_base.cc:1061] running on GCE node
I20260806 07:55:53.553076 28574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:53.554237 28574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:53.555514 28574 hybrid_clock.cc:648] HybridClock initialized: now 1786002953555460 us; error 39 us; skew 500 ppm
I20260806 07:55:53.555943 28574 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:53 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002953, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:53.559441 28574 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:53.561148 28574 webserver.cc:533] Webserver started at http://127.20.202.193:35203/ using document root <none> and password file <none>
I20260806 07:55:53.561934 28574 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:53.562041 28574 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:53.562322 28574 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:53.564774 28574 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:53.565557 28574 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:53.571403 28574 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:55:53.575240 28590 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:53.576731 28574 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:53.576870 28574 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:53.577039 28574 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:53.596155 28574 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:53.600443 28574 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:53.600687 28574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:53.601660 28574 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:53.603308 28574 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:53.603380 28574 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:53.603462 28574 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:53.603505 28574 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:53.639590 28574 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:34249
I20260806 07:55:53.639629 28704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:34249 every 8 connection(s)
I20260806 07:55:53.641019 28574 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:53.649931 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28574
I20260806 07:55:53.650208 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:55:53 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002953, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:55:53 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002953, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:53 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002953, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:53.687515 28708 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45847
I20260806 07:55:53.687642 28707 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35425
I20260806 07:55:53.687996 28707 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:53.687997 28708 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:53.689230 28707 heartbeater.cc:507] Master 127.20.202.254:35425 requested a full tablet report, sending...
I20260806 07:55:53.689258 28708 heartbeater.cc:507] Master 127.20.202.253:45847 requested a full tablet report, sending...
I20260806 07:55:53.691816 28313 ts_manager.cc:194] Registered new tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:53.691887 28390 ts_manager.cc:194] Registered new tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:53.695298 28706 heartbeater.cc:344] Connected to a master server at 127.20.202.252:40393
I20260806 07:55:53.695441 28706 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:53.695976 28706 heartbeater.cc:507] Master 127.20.202.252:40393 requested a full tablet report, sending...
I20260806 07:55:53.697680 28467 ts_manager.cc:194] Registered new tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:53.699298 28467 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:51059
I20260806 07:55:53.710404 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:55:53.863375 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:32809
--webserver_interface=127.20.202.251
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:54.228914 28720 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:54.229315 28720 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:54.229485 28720 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:54.238535 28720 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:54.238655 28720 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:54.238722 28720 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:54.238776 28720 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:54.250735 28720 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:32809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:54.252429 28720 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:54.253906 28720 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:54.263048 28725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:54.263096 28726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:54.263736 28728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:54.264071 28720 server_base.cc:1061] running on GCE node
I20260806 07:55:54.264839 28720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:54.266356 28720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:54.267590 28720 hybrid_clock.cc:648] HybridClock initialized: now 1786002954267543 us; error 55 us; skew 500 ppm
I20260806 07:55:54.267933 28720 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002954, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:54.271276 28720 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:55:54.272945 28720 webserver.cc:533] Webserver started at http://127.20.202.251:37975/ using document root <none> and password file <none>
I20260806 07:55:54.273700 28720 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:54.273800 28720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:54.274101 28720 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:55:54.276556 28720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "bddf58a68ab54049ad5a55cf97c379b0"
format_stamp: "Formatted at 2026-08-06 07:55:54 on dist-test-slave-g350"
I20260806 07:55:54.277345 28720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "bddf58a68ab54049ad5a55cf97c379b0"
format_stamp: "Formatted at 2026-08-06 07:55:54 on dist-test-slave-g350"
I20260806 07:55:54.283344 28720 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260806 07:55:54.287712 28735 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:54.289387 28720 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:55:54.289566 28720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "bddf58a68ab54049ad5a55cf97c379b0"
format_stamp: "Formatted at 2026-08-06 07:55:54 on dist-test-slave-g350"
I20260806 07:55:54.289791 28720 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:54.310117 28720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:54.314566 28720 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:54.314819 28720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:54.336427 28720 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:32809
I20260806 07:55:54.336478 28786 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:32809 every 8 connection(s)
I20260806 07:55:54.338150 28720 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:54.342770 28787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:55:54.347702 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28720
I20260806 07:55:54.348869 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002954, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:55:54.371462 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002954, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:54.386792 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002954, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:54.400229 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002954, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:55:54.418216 28787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Bootstrap starting.
I20260806 07:55:54.421512 28787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Neither blocks nor log segments found. Creating new log.
I20260806 07:55:54.422770 28787 log.cc:826] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:54.426102 28787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: No bootstrap required, opened a new log
I20260806 07:55:54.431468 28787 raft_consensus.cc:359] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } }
I20260806 07:55:54.431831 28787 raft_consensus.cc:385] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:55:54.431948 28787 raft_consensus.cc:740] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddf58a68ab54049ad5a55cf97c379b0, State: Initialized, Role: FOLLOWER
I20260806 07:55:54.432586 28787 consensus_queue.cc:260] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } }
I20260806 07:55:54.433560 28796 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } } }
I20260806 07:55:54.434019 28796 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:54.434696 28787 sys_catalog.cc:565] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:55:54.444898 28807 catalog_manager.cc:1594] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:55:54.445075 28807 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:55:54.453004 28720 master_runner.cc:429] Detected that this master bddf58a68ab54049ad5a55cf97c379b0 is joining an existing cluster
I20260806 07:55:54.453163 28720 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:55:54.454151 28720 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:55:54.454578 28720 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:55:54.454795 28720 tablet_replica.cc:333] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: stopping tablet replica
I20260806 07:55:54.456868 28720 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:32809
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:55:54.516911 28467 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:32809: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 32809
}
attrs {
promote: true
}
}
dest_uuid: "5a612facfe31495c9fc2d20376ca161a"
cas_config_opid_index: -1
I20260806 07:55:54.518388 28467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } }
I20260806 07:55:54.521659 28322 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Refusing update from remote peer 5a612facfe31495c9fc2d20376ca161a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:55:54.522830 28578 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260806 07:55:54.523466 28454 consensus_peers.cc:597] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a -> Peer bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251:32809): Couldn't send request to peer bddf58a68ab54049ad5a55cf97c379b0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:54.524813 28399 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 1 FOLLOWER]: Refusing update from remote peer 5a612facfe31495c9fc2d20376ca161a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:55:54.525574 28818 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:55:54.526716 28578 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.527335 28322 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.531176 28578 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:32809
I20260806 07:55:54.531915 28720 master.cc:562] Master@127.20.202.251:32809 shutting down...
I20260806 07:55:54.535218 28578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.535288 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.535753 28578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:54.535818 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:54.539495 28399 raft_consensus.cc:2955] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.544984 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.545250 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:54.546345 28824 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.546712 28824 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:54.546816 28825 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.547262 28825 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:54.578646 28720 master.cc:584] Master@127.20.202.251:32809 shutdown complete.
I20260806 07:55:54.578745 28720 master_runner.cc:306] Clearing existing system tablet
I20260806 07:55:54.579246 28720 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:54.579372 28720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:54.582095 28720 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 07:55:54.583808 28827 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:54.584196 28720 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:54.584504 28720 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 07:55:54.584640 28720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "bddf58a68ab54049ad5a55cf97c379b0"
format_stamp: "Formatted at 2026-08-06 07:55:54 on dist-test-slave-g350"
I20260806 07:55:54.589838 28720 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:55:54.597779 28720 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:55:54.597999 28720 log.cc:1199] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:55:54.598541 28720 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Deleting consensus metadata
I20260806 07:55:54.598891 28720 master_runner.cc:316] Copying system tablet from 127.20.202.252:40393
I20260806 07:55:54.601980 28720 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.252:40393
I20260806 07:55:54.616819 28488 tablet_copy_service.cc:140] P 5a612facfe31495c9fc2d20376ca161a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bddf58a68ab54049ad5a55cf97c379b0 ({username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:40862)
I20260806 07:55:54.617183 28488 tablet_copy_service.cc:161] P 5a612facfe31495c9fc2d20376ca161a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bddf58a68ab54049ad5a55cf97c379b0 at {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:40862: session id = bddf58a68ab54049ad5a55cf97c379b0-00000000000000000000000000000000
I20260806 07:55:54.621945 28488 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:55:54.624513 28720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:55:54.632200 28720 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: tablet copy: Starting download of 0 data blocks...
I20260806 07:55:54.632540 28720 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: tablet copy: Starting download of 1 WAL segments...
I20260806 07:55:54.636101 28720 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:55:54.641165 28488 tablet_copy_service.cc:342] P 5a612facfe31495c9fc2d20376ca161a: Request end of tablet copy session bddf58a68ab54049ad5a55cf97c379b0-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:40862
I20260806 07:55:54.641427 28488 tablet_copy_service.cc:434] P 5a612facfe31495c9fc2d20376ca161a: ending tablet copy session bddf58a68ab54049ad5a55cf97c379b0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bddf58a68ab54049ad5a55cf97c379b0
W20260806 07:55:54.649165 28720 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:55:54.652549 28720 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:54.655448 28835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:54.655529 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:54.659747 28837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:54.659770 28720 server_base.cc:1061] running on GCE node
I20260806 07:55:54.660319 28720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:54.661021 28720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:54.662266 28720 hybrid_clock.cc:648] HybridClock initialized: now 1786002954662197 us; error 47 us; skew 500 ppm
I20260806 07:55:54.662531 28720 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002954, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:54.665262 28720 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:55:54.668984 28720 webserver.cc:533] Webserver started at http://127.20.202.251:37471/ using document root <none> and password file <none>
I20260806 07:55:54.669338 28720 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:54.669443 28720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:54.673499 28720 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 07:55:54.675743 28844 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:54.676198 28720 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 07:55:54.676338 28720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "bddf58a68ab54049ad5a55cf97c379b0"
format_stamp: "Formatted at 2026-08-06 07:55:54 on dist-test-slave-g350"
I20260806 07:55:54.676497 28720 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:54.701239 28720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:54.702009 28706 heartbeater.cc:499] Master 127.20.202.252:40393 was elected leader, sending a full tablet report...
I20260806 07:55:54.705379 28720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:54.726234 28720 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:32809
I20260806 07:55:54.726279 28896 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:32809 every 8 connection(s)
I20260806 07:55:54.727344 28720 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:55:54.731436 28897 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:54.733965 28897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Bootstrap starting.
I20260806 07:55:54.752065 28897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:54.752724 28897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: Bootstrap complete.
I20260806 07:55:54.754020 28897 raft_consensus.cc:359] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } }
I20260806 07:55:54.754328 28897 raft_consensus.cc:740] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bddf58a68ab54049ad5a55cf97c379b0, State: Initialized, Role: LEARNER
I20260806 07:55:54.754638 28897 consensus_queue.cc:260] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } }
I20260806 07:55:54.755764 28901 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:54.756124 28897 sys_catalog.cc:565] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:54.756229 28901 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:55:54.765130 28912 catalog_manager.cc:1295] Loaded cluster ID: c38fcac4dc1b413b83f9cf05454e9f3b
I20260806 07:55:54.765230 28912 catalog_manager.cc:1588] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: loading cluster ID for follower catalog manager: success
I20260806 07:55:54.768143 28912 catalog_manager.cc:1610] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: acquiring CA information for follower catalog manager: success
I20260806 07:55:54.770539 28912 catalog_manager.cc:1638] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:54.808527 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28574
Aug 06 07:55:54 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:55:54.829989 28700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:54.920850 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28574
I20260806 07:55:54.941084 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:34249
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--webserver_port=35203
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:55.118862 28872 raft_consensus.cc:1217] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:55:55.119607 28901 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a612facfe31495c9fc2d20376ca161a. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:55.119837 28901 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:55:55.300011 28915 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:55.300395 28915 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:55.300563 28915 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:55.309056 28915 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:55.309317 28915 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:55.320256 28915 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:34249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35203
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:55.322005 28915 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:55.323406 28915 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:55.333432 28922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:55.333953 28924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:55.333585 28921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:55.334618 28915 server_base.cc:1061] running on GCE node
I20260806 07:55:55.335279 28915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:55.336344 28915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:55.337677 28915 hybrid_clock.cc:648] HybridClock initialized: now 1786002955337560 us; error 64 us; skew 500 ppm
I20260806 07:55:55.338052 28915 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:55 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002955, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:55.341550 28915 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:55.343360 28915 webserver.cc:533] Webserver started at http://127.20.202.193:35203/ using document root <none> and password file <none>
I20260806 07:55:55.344079 28915 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:55.344194 28915 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:55.350396 28915 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:55:55.353636 28931 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:55.355383 28915 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:55:55.355625 28915 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:55.356477 28915 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:55.371379 28915 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:55.376091 28915 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:55.376415 28915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:55.377677 28915 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:55.379432 28915 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:55.379540 28915 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:55.379705 28915 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:55.379803 28915 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:55.420102 28915 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:34249
I20260806 07:55:55.420162 29046 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:34249 every 8 connection(s)
I20260806 07:55:55.421887 28915 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:55.425491 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 28915
Aug 06 07:55:55 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002955, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:55:55 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002955, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:55:55 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002955, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:55:55.468788 29049 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35425
I20260806 07:55:55.469275 29049 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:55.470503 29047 heartbeater.cc:344] Connected to a master server at 127.20.202.251:32809
I20260806 07:55:55.470554 29049 heartbeater.cc:507] Master 127.20.202.254:35425 requested a full tablet report, sending...
I20260806 07:55:55.470667 29047 heartbeater.cc:461] Registering TS with master...
Aug 06 07:55:55 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002955, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:55:55.471227 29047 heartbeater.cc:507] Master 127.20.202.251:32809 requested a full tablet report, sending...
I20260806 07:55:55.473847 28862 ts_manager.cc:194] Registered new tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:55.474352 28313 ts_manager.cc:194] Re-registered known tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:55.475041 29048 heartbeater.cc:344] Connected to a master server at 127.20.202.252:40393
I20260806 07:55:55.475231 29048 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:55.475997 29048 heartbeater.cc:507] Master 127.20.202.252:40393 requested a full tablet report, sending...
I20260806 07:55:55.477582 28467 ts_manager.cc:194] Re-registered known tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:55.478974 28467 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:40415
I20260806 07:55:55.481797 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:55.483451 29050 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45847
I20260806 07:55:55.483628 29050 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:55.484076 29050 heartbeater.cc:507] Master 127.20.202.253:45847 requested a full tablet report, sending...
I20260806 07:55:55.486398 28390 ts_manager.cc:194] Re-registered known tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:55.492401 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:55:55.492599 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28281
I20260806 07:55:55.493736 28344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:55.625720 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28281
I20260806 07:55:55.650931 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35425
--webserver_interface=127.20.202.254
--webserver_port=38037
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:55.663094 28822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:55.663421 28822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:55.686162 29062 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a: attempting to promote NON_VOTER bddf58a68ab54049ad5a55cf97c379b0 to VOTER
I20260806 07:55:55.687433 29062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } }
I20260806 07:55:55.690671 28399 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 1 FOLLOWER]: Refusing update from remote peer 5a612facfe31495c9fc2d20376ca161a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:55.690846 28872 raft_consensus.cc:1275] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 1 LEARNER]: Refusing update from remote peer 5a612facfe31495c9fc2d20376ca161a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:55:55.691516 29062 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:55:55.691900 29061 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:55:55.707610 28452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.254:35425: connect: Connection refused (error 111)
W20260806 07:55:55.709985 28452 consensus_peers.cc:597] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a -> Peer b74cd3d7a919435d9fb276f1db033d08 (127.20.202.254:35425): Couldn't send request to peer b74cd3d7a919435d9fb276f1db033d08. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:35425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:55.711081 29062 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.711905 28399 raft_consensus.cc:2955] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.715919 28872 raft_consensus.cc:2955] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.725940 29070 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.726300 29070 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff9b89dfc944ce5b093eae9b10075fe [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:55.733314 29071 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.733733 29071 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddf58a68ab54049ad5a55cf97c379b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:55.734313 29061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.734619 29061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:55.734992 29061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.735092 29062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:55.735250 29061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
I20260806 07:55:55.735327 29062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
W20260806 07:55:56.058828 29060 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:56.059178 29060 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:56.059268 29060 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:56.068262 29060 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:55:56.068370 29060 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:56.068427 29060 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:55:56.068482 29060 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:55:56.079726 29060 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=38037
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:56.081480 29060 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:56.082888 29060 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:56.091228 29076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:56.092145 29077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:56.093747 29079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:56.094358 29060 server_base.cc:1061] running on GCE node
I20260806 07:55:56.095229 29060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:56.096755 29060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:56.098003 29060 hybrid_clock.cc:648] HybridClock initialized: now 1786002956097940 us; error 40 us; skew 500 ppm
I20260806 07:55:56.098476 29060 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:56.102115 29060 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:55:56.103893 29060 webserver.cc:533] Webserver started at http://127.20.202.254:38037/ using document root <none> and password file <none>
I20260806 07:55:56.104602 29060 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:56.104713 29060 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:56.110642 29060 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:55:56.113668 29087 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:56.115316 29060 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:55:56.115579 29060 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "b74cd3d7a919435d9fb276f1db033d08"
format_stamp: "Formatted at 2026-08-06 07:55:50 on dist-test-slave-g350"
I20260806 07:55:56.116479 29060 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:56.133229 29060 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:56.137992 29060 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:56.138301 29060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:56.163467 29060 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:35425
I20260806 07:55:56.163479 29139 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:35425 every 8 connection(s)
I20260806 07:55:56.165335 29060 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:55:56.172084 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29060
I20260806 07:55:56.172438 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28915
I20260806 07:55:56.172902 29140 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:55:56.178010 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Bootstrap starting.
I20260806 07:55:56.191414 29042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:56.200356 29140 log.cc:826] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Log is configured to *not* fsync() on all Append() calls
I20260806 07:55:56.213109 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:55:56.213866 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: Bootstrap complete.
I20260806 07:55:56.224632 29140 raft_consensus.cc:359] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } }
I20260806 07:55:56.225728 29140 raft_consensus.cc:740] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74cd3d7a919435d9fb276f1db033d08, State: Initialized, Role: FOLLOWER
I20260806 07:55:56.226632 29140 consensus_queue.cc:260] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } }
I20260806 07:55:56.227486 29146 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: true } } }
I20260806 07:55:56.227804 29146 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:56.228825 29140 sys_catalog.cc:565] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:55:56.253417 29157 catalog_manager.cc:1295] Loaded cluster ID: c38fcac4dc1b413b83f9cf05454e9f3b
I20260806 07:55:56.253638 29157 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: loading cluster ID for follower catalog manager: success
I20260806 07:55:56.258188 29157 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: acquiring CA information for follower catalog manager: success
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:56.261260 29157 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:55:56.328502 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28915
I20260806 07:55:56.328917 29115 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bddf58a68ab54049ad5a55cf97c379b0 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:56.338893 29167 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:56.339072 29146 sys_catalog.cc:455] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a612facfe31495c9fc2d20376ca161a. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:56.339406 29167 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:56.339459 29146 sys_catalog.cc:458] T 00000000000000000000000000000000 P b74cd3d7a919435d9fb276f1db033d08 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:55:56.360186 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:34249
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--webserver_port=35203
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:55:56.714478 29069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a612facfe31495c9fc2d20376ca161a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b74cd3d7a919435d9fb276f1db033d08" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35425 } } peers { permanent_uuid: "bff9b89dfc944ce5b093eae9b10075fe" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 45847 } } peers { permanent_uuid: "5a612facfe31495c9fc2d20376ca161a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40393 } } peers { permanent_uuid: "bddf58a68ab54049ad5a55cf97c379b0" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 32809 } attrs { promote: false } } }
I20260806 07:55:56.714932 29069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a [sys.catalog]: This master's current role is: LEADER
W20260806 07:55:56.728338 29168 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:56.728689 29168 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:56.728868 29168 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:56.737571 29168 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:56.737891 29168 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:56.749388 29168 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:34249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35203
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:56.751148 29168 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:56.752561 29168 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:56.763765 29176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:56.764226 29179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:56.764462 29177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:56.765175 29168 server_base.cc:1061] running on GCE node
I20260806 07:55:56.765905 29168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:56.766949 29168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:56.768195 29168 hybrid_clock.cc:648] HybridClock initialized: now 1786002956768141 us; error 41 us; skew 500 ppm
I20260806 07:55:56.768539 29168 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:56.772441 29168 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:56.774479 29168 webserver.cc:533] Webserver started at http://127.20.202.193:35203/ using document root <none> and password file <none>
I20260806 07:55:56.775251 29168 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:56.775352 29168 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:56.781313 29168 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:55:56.784350 29186 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:56.786024 29168 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:55:56.786180 29168 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:56.786891 29168 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:56.829571 29168 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:56.834139 29168 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:56.834440 29168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:56.835485 29168 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:56.837222 29168 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:56.837355 29168 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:56.837481 29168 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:56.837571 29168 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:56.875250 29168 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:34249
I20260806 07:55:56.875273 29300 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:34249 every 8 connection(s)
I20260806 07:55:56.877074 29168 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:56.885946 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29168
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:55:56.911831 29302 heartbeater.cc:344] Connected to a master server at 127.20.202.251:32809
I20260806 07:55:56.911975 29305 heartbeater.cc:344] Connected to a master server at 127.20.202.253:45847
I20260806 07:55:56.912328 29302 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:56.912333 29305 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:56.913661 29305 heartbeater.cc:507] Master 127.20.202.253:45847 requested a full tablet report, sending...
I20260806 07:55:56.913679 29302 heartbeater.cc:507] Master 127.20.202.251:32809 requested a full tablet report, sending...
I20260806 07:55:56.916085 28862 ts_manager.cc:194] Re-registered known tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:56.916224 28389 ts_manager.cc:194] Re-registered known tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
Aug 06 07:55:56 dist-test-slave-g350 krb5kdc[28264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:55:56.922111 29304 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35425
I20260806 07:55:56.922302 29304 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:56.922844 29304 heartbeater.cc:507] Master 127.20.202.254:35425 requested a full tablet report, sending...
I20260806 07:55:56.924851 29104 ts_manager.cc:194] Registered new tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:56.926085 29303 heartbeater.cc:344] Connected to a master server at 127.20.202.252:40393
I20260806 07:55:56.926252 29303 heartbeater.cc:461] Registering TS with master...
I20260806 07:55:56.926777 29303 heartbeater.cc:507] Master 127.20.202.252:40393 requested a full tablet report, sending...
I20260806 07:55:56.928020 28467 ts_manager.cc:194] Re-registered known tserver with Master: 6938c51e4df64027ac40ae504c81ace7 (127.20.202.193:34249)
I20260806 07:55:56.929255 28467 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:40101
I20260806 07:55:56.967183 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29168
W20260806 07:55:56.968619 29319 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:56.964707 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.20.202.193:34249)
0806 07:55:56.965543 (+ 836us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:56.965553 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:56.965586 (+ 33us) server_negotiation.cc:408] Connection header received
0806 07:55:56.965720 (+ 134us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:56.965728 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:56.965958 (+ 230us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:56.966140 (+ 182us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:56.967125 (+ 985us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37630: BlockingRecv error: recv got EOF from 127.0.0.1:37630 (error 108)
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":509,"threads_started":1}
W20260806 07:55:56.969923 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29168: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:34249: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:55:56.970029 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29168
I20260806 07:55:56.987658 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:34249
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--webserver_port=35203
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:57.347721 29321 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:57.348109 29321 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:57.348265 29321 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:55:57.348358 29321 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:57.357040 29321 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:57.357308 29321 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:57.368698 29321 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:34249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35203
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:57.370538 29321 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:57.371975 29321 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:57.382594 29327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:57.382562 29326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:57.383363 29329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:57.383564 29321 server_base.cc:1061] running on GCE node
I20260806 07:55:57.384280 29321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:57.385430 29321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:57.386696 29321 hybrid_clock.cc:648] HybridClock initialized: now 1786002957386634 us; error 63 us; skew 500 ppm
I20260806 07:55:57.386981 29321 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:57 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002957, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:57.390604 29321 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:57.392246 29321 webserver.cc:533] Webserver started at http://127.20.202.193:35203/ using document root <none> and password file <none>
I20260806 07:55:57.393018 29321 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:57.393114 29321 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:57.399499 29321 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:55:57.403038 29336 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:57.405025 29321 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:55:57.405563 29321 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:57.406488 29321 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:57.428690 29321 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:57.436277 29321 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:57.436532 29321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:57.437500 29321 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:57.439333 29321 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:57.439409 29321 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:57.439491 29321 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:57.439534 29321 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:55:57.439867 29321 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:55:57.439926 29321 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:55:57.440026 29321 fault_injection.cc:59] injecting fault for kudu (pid 29321): FLAGS_fault_before_start (process will exit)
I20260806 07:55:57.462469 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:34249
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393
--webserver_port=35203
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:55:57.842720 29346 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:55:57.843122 29346 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:55:57.843281 29346 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:55:57.852042 29346 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:55:57.852322 29346 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:55:57.863854 29346 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:34249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=35203
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35425,127.20.202.253:45847,127.20.202.252:40393,127.20.202.251:32809
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:55:57.865686 29346 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:55:57.867323 29346 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:55:57.877712 29351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:57.877884 29352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:55:57.878576 29354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:55:57.879120 29346 server_base.cc:1061] running on GCE node
I20260806 07:55:57.879705 29346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:55:57.880759 29346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:55:57.882037 29346 hybrid_clock.cc:648] HybridClock initialized: now 1786002957881955 us; error 64 us; skew 500 ppm
I20260806 07:55:57.882400 29346 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:55:57 dist-test-slave-g350 krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002957, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:55:57.886046 29346 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:55:57.887774 29346 webserver.cc:533] Webserver started at http://127.20.202.193:35203/ using document root <none> and password file <none>
I20260806 07:55:57.888619 29346 fs_manager.cc:362] Metadata directory not provided
I20260806 07:55:57.888729 29346 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:55:57.895172 29346 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:55:57.898547 29361 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:57.900167 29346 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:55:57.900331 29346 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6938c51e4df64027ac40ae504c81ace7"
format_stamp: "Formatted at 2026-08-06 07:55:53 on dist-test-slave-g350"
I20260806 07:55:57.901118 29346 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:55:57.941274 29346 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:55:57.946892 29346 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:55:57.947211 29346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:55:57.950546 29346 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:55:57.953068 29346 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:55:57.953220 29346 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:57.953354 29346 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:55:57.953446 29346 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:55:57.994014 29346 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:34249
I20260806 07:55:57.994118 29476 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:34249 every 8 connection(s)
I20260806 07:55:57.995784 29346 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:55:58.001828 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29346
I20260806 07:55:58.002146 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29346
W20260806 07:55:58.027467 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29346: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:34249: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:55:58.027619 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29346
I20260806 07:55:58.047415 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29060
W20260806 07:55:58.048225 29482 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:57.999849 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:46563 (local address 127.20.202.254:35425)
0806 07:55:58.000365 (+ 516us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:58.000372 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:58.002267 (+ 1895us) server_negotiation.cc:408] Connection header received
0806 07:55:58.005642 (+ 3375us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:58.005652 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:58.005763 (+ 111us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:58.005875 (+ 112us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:58.011257 (+ 5382us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:55:58.013253 (+ 1996us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:55:58.017028 (+ 3775us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:55:58.017678 (+ 650us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:55:58.047257 (+ 29579us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:46563: BlockingRecv error: recv got EOF from 127.20.202.193:46563 (error 108)
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":256,"threads_started":1}
W20260806 07:55:58.048220 29483 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:57.999868 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:46555 (local address 127.20.202.251:32809)
0806 07:55:58.000539 (+ 671us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:58.000546 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:58.003593 (+ 3047us) server_negotiation.cc:408] Connection header received
0806 07:55:58.005394 (+ 1801us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:58.005402 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:58.005525 (+ 123us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:58.005685 (+ 160us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:58.009725 (+ 4040us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:55:58.011407 (+ 1682us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:55:58.047206 (+ 35799us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:46555: BlockingRecv error: recv got EOF from 127.20.202.193:46555 (error 108)
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":902,"threads_started":1}
W20260806 07:55:58.048259 29491 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:58.018207 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:33617 (local address 127.20.202.253:45847)
0806 07:55:58.030200 (+ 11993us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:58.030209 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:58.047167 (+ 16958us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:33617: BlockingRecv error: recv got EOF from 127.20.202.193:33617 (error 108)
Metrics: {"server-negotiator.queue_time_us":11839,"thread_start_us":11705,"threads_started":1}
W20260806 07:55:58.049733 29489 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:58.004102 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:42141 (local address 127.20.202.252:40393)
0806 07:55:58.019937 (+ 15835us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:58.019946 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:58.019974 (+ 28us) server_negotiation.cc:408] Connection header received
0806 07:55:58.020057 (+ 83us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:58.020066 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:58.020168 (+ 102us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:58.020326 (+ 158us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:58.022825 (+ 2499us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:55:58.024659 (+ 1834us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:55:58.027264 (+ 2605us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:55:58.029117 (+ 1853us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:55:58.049011 (+ 19894us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:42141: BlockingRecv error: recv got EOF from 127.20.202.193:42141 (error 108)
Metrics: {"server-negotiator.queue_time_us":15657,"thread_start_us":18528,"threads_started":1}
W20260806 07:55:58.050118 29482 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:55:58.048609 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.20.202.254:35425)
0806 07:55:58.048768 (+ 159us) server_negotiation.cc:207] Beginning negotiation
0806 07:55:58.048774 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:55:58.048795 (+ 21us) server_negotiation.cc:408] Connection header received
0806 07:55:58.049003 (+ 208us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:55:58.049009 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:55:58.049079 (+ 70us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:55:58.049192 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:55:58.049924 (+ 732us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42724: BlockingRecv error: recv got EOF from 127.0.0.1:42724 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 07:55:58.050096 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29060: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:35425: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:55:58.050221 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29060
I20260806 07:55:58.071524 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28358
I20260806 07:55:58.072858 28421 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:55:58.162779 28452 consensus_peers.cc:597] T 00000000000000000000000000000000 P 5a612facfe31495c9fc2d20376ca161a -> Peer b74cd3d7a919435d9fb276f1db033d08 (127.20.202.254:35425): Couldn't send request to peer b74cd3d7a919435d9fb276f1db033d08. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:35425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:55:58.187299 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28358
I20260806 07:55:58.211071 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28436
I20260806 07:55:58.212340 28498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:58.331369 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28436
I20260806 07:55:58.355252 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 28720
I20260806 07:55:58.356675 28892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:55:58.473527 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 28720
2026-08-06T07:55:58Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/7 (10431 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/8
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:55:58 dist-test-slave-g350 krb5kdc[29499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:55:58 dist-test-slave-g350 krb5kdc[29499](info): set up 2 sockets
Aug 06 07:55:58 dist-test-slave-g350 krb5kdc[29499](info): commencing operation
krb5kdc: starting...
W20260806 07:56:00.590626 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.070s user 0.002s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:56:00 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:56:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:56:00Z Disabled control of system clock
2026-08-06T07:56:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:56:00Z Disabled control of system clock
2026-08-06T07:56:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:56:00Z Disabled control of system clock
2026-08-06T07:56:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:56:00Z Disabled control of system clock
2026-08-06T07:56:01Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:56:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:01.187351 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45617
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:01.556051 29523 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:01.556456 29523 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:01.556627 29523 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:01.565485 29523 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:01.565631 29523 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:01.565738 29523 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:01.565825 29523 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:01.577383 29523 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:01.579221 29523 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:01.580753 29523 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:01.589697 29528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:01.589748 29529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:01.590651 29531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:01.591012 29523 server_base.cc:1061] running on GCE node
I20260806 07:56:01.591786 29523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:01.593407 29523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:01.594686 29523 hybrid_clock.cc:648] HybridClock initialized: now 1786002961594522 us; error 130 us; skew 500 ppm
I20260806 07:56:01.595000 29523 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:01 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002961, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:01.598321 29523 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:56:01.600248 29523 webserver.cc:533] Webserver started at http://127.20.202.254:41401/ using document root <none> and password file <none>
I20260806 07:56:01.601045 29523 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:01.601147 29523 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:01.601543 29523 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:01.604111 29523 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "8bbe00a5e948468e8914c0c9208a91c2"
format_stamp: "Formatted at 2026-08-06 07:56:01 on dist-test-slave-g350"
I20260806 07:56:01.604991 29523 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "8bbe00a5e948468e8914c0c9208a91c2"
format_stamp: "Formatted at 2026-08-06 07:56:01 on dist-test-slave-g350"
I20260806 07:56:01.611088 29523 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 07:56:01.614982 29538 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:01.616526 29523 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:56:01.616719 29523 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "8bbe00a5e948468e8914c0c9208a91c2"
format_stamp: "Formatted at 2026-08-06 07:56:01 on dist-test-slave-g350"
I20260806 07:56:01.616937 29523 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:01.639343 29523 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:01.643512 29523 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:01.643810 29523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:01.665899 29523 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45617
I20260806 07:56:01.665935 29590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45617 every 8 connection(s)
I20260806 07:56:01.667485 29523 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:56:01.670642 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29523
I20260806 07:56:01.670878 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:56:01.672209 29591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:01.678462 29591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:01 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002961, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
I20260806 07:56:01.703065 29591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
W20260806 07:56:01.704732 29539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34107: connect: Connection refused (error 111)
W20260806 07:56:01.706809 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34107: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34107: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:01.756203 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } attempt: 1
W20260806 07:56:01.758688 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34107: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34107: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:01.810532 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34107
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:01.827008 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } attempt: 2
W20260806 07:56:01.829358 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34107: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34107: connect: Connection refused (error 111)
I20260806 07:56:01.920704 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } attempt: 3
W20260806 07:56:01.923066 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34107: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34107: connect: Connection refused (error 111)
I20260806 07:56:02.066375 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } attempt: 4
W20260806 07:56:02.068696 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:34107: Network error: Client connection negotiation failed: client connection to 127.20.202.253:34107: connect: Connection refused (error 111)
W20260806 07:56:02.173070 29600 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:02.173401 29600 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:02.173609 29600 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:02.182188 29600 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:02.182296 29600 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:02.182396 29600 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:02.182487 29600 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:02.193423 29600 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:34107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:02.195150 29600 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:02.196583 29600 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:02.205504 29606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:02.205555 29607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:02.206142 29609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:02.206724 29600 server_base.cc:1061] running on GCE node
I20260806 07:56:02.207515 29600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:02.209138 29600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:02.210474 29600 hybrid_clock.cc:648] HybridClock initialized: now 1786002962210292 us; error 149 us; skew 500 ppm
I20260806 07:56:02.210839 29600 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:02.214083 29600 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:56:02.215842 29600 webserver.cc:533] Webserver started at http://127.20.202.253:39215/ using document root <none> and password file <none>
I20260806 07:56:02.216565 29600 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:02.216692 29600 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:02.217053 29600 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:02.219569 29600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "b5e46660ad00497fa221e25ce7f4d004"
format_stamp: "Formatted at 2026-08-06 07:56:02 on dist-test-slave-g350"
I20260806 07:56:02.220396 29600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "b5e46660ad00497fa221e25ce7f4d004"
format_stamp: "Formatted at 2026-08-06 07:56:02 on dist-test-slave-g350"
I20260806 07:56:02.226073 29600 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 07:56:02.229436 29616 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:02.230880 29600 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:02.231060 29600 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "b5e46660ad00497fa221e25ce7f4d004"
format_stamp: "Formatted at 2026-08-06 07:56:02 on dist-test-slave-g350"
I20260806 07:56:02.231257 29600 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:02.247485 29600 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:02.251545 29600 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:02.251844 29600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:02.274274 29600 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:34107
I20260806 07:56:02.274293 29667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:34107 every 8 connection(s)
I20260806 07:56:02.275964 29600 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:56:02.280375 29668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:02.284860 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29600
I20260806 07:56:02.285058 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:56:02.286372 29668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:02.310288 29668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
I20260806 07:56:02.328538 29668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:02.329852 29619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
W20260806 07:56:02.331799 29668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:02.367990 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } attempt: 5
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002961, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:02.381158 29668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 1
W20260806 07:56:02.383543 29668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
I20260806 07:56:02.383723 29591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:02.385790 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:02.433178 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33431
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:02.437043 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 1
W20260806 07:56:02.439332 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
I20260806 07:56:02.451787 29668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 2
W20260806 07:56:02.454304 29668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
I20260806 07:56:02.507597 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 2
W20260806 07:56:02.510373 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
I20260806 07:56:02.545631 29668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 3
W20260806 07:56:02.547904 29668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
I20260806 07:56:02.616640 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 3
W20260806 07:56:02.619028 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
I20260806 07:56:02.691161 29668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 4
W20260806 07:56:02.693292 29668 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
W20260806 07:56:02.795053 29678 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:02.795389 29678 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:02.795485 29678 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:56:02.796283 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 4
W20260806 07:56:02.798517 29591 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:33431: Network error: Client connection negotiation failed: client connection to 127.20.202.252:33431: connect: Connection refused (error 111)
W20260806 07:56:02.804929 29678 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:02.805029 29678 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:02.805121 29678 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:02.805184 29678 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:02.816529 29678 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:33431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:02.818353 29678 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:02.819790 29678 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:02.828348 29683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:02.828438 29684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:02.829030 29686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:02.830106 29678 server_base.cc:1061] running on GCE node
I20260806 07:56:02.830904 29678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:02.832664 29678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:02.833950 29678 hybrid_clock.cc:648] HybridClock initialized: now 1786002962833807 us; error 102 us; skew 500 ppm
I20260806 07:56:02.834334 29678 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:02.838016 29678 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:56:02.839636 29678 webserver.cc:533] Webserver started at http://127.20.202.252:35697/ using document root <none> and password file <none>
I20260806 07:56:02.840373 29678 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:02.840503 29678 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:02.840854 29678 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:02.843389 29678 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "821e81a3b63c423d84ff6c8d52e540b4"
format_stamp: "Formatted at 2026-08-06 07:56:02 on dist-test-slave-g350"
I20260806 07:56:02.844223 29678 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "821e81a3b63c423d84ff6c8d52e540b4"
format_stamp: "Formatted at 2026-08-06 07:56:02 on dist-test-slave-g350"
I20260806 07:56:02.850030 29678 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 07:56:02.853705 29693 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:02.855295 29678 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:56:02.855497 29678 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "821e81a3b63c423d84ff6c8d52e540b4"
format_stamp: "Formatted at 2026-08-06 07:56:02 on dist-test-slave-g350"
I20260806 07:56:02.855737 29678 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:02.870851 29678 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:02.878644 29678 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:02.878963 29678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:02.901427 29678 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:33431
I20260806 07:56:02.901446 29745 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:33431 every 8 connection(s)
I20260806 07:56:02.903213 29678 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:56:02.907050 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29678
I20260806 07:56:02.907306 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:56:02.908134 29747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:02.916669 29747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:02.939814 29747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:02.957998 29747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:02.977183 29747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: Bootstrap starting.
I20260806 07:56:02.980496 29747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:02.981760 29747 log.cc:826] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:02.984642 29747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: No bootstrap required, opened a new log
I20260806 07:56:02.989914 29747 raft_consensus.cc:359] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:02.990214 29747 raft_consensus.cc:385] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:02.990299 29747 raft_consensus.cc:740] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821e81a3b63c423d84ff6c8d52e540b4, State: Initialized, Role: FOLLOWER
I20260806 07:56:02.991001 29747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:02.992553 29668 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 5
I20260806 07:56:02.992904 29758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:02.993278 29758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:02.994340 29747 sys_catalog.cc:565] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 07:56:02 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
W20260806 07:56:03.008697 29769 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:03.008842 29769 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:03.005450 29678 master_runner.cc:187] Error getting master registration for 127.20.202.254:45617: OK, instance_id { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" instance_seqno: 1786002961659961 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:56:03.013955 29678 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:56:03.014024 29668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: Bootstrap starting.
I20260806 07:56:03.018282 29668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:03.019646 29668 log.cc:826] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:03.022810 29668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: No bootstrap required, opened a new log
I20260806 07:56:03.030576 29668 raft_consensus.cc:359] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.030936 29668 raft_consensus.cc:385] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:03.031075 29668 raft_consensus.cc:740] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e46660ad00497fa221e25ce7f4d004, State: Initialized, Role: FOLLOWER
I20260806 07:56:03.031833 29668 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.032766 29772 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.033041 29772 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:03.034235 29668 sys_catalog.cc:565] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:56:03.046592 29783 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:03.046727 29783 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:03.047140 29772 raft_consensus.cc:493] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:56:03.047389 29772 raft_consensus.cc:515] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.049300 29772 leader_election.cc:290] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bbe00a5e948468e8914c0c9208a91c2 (127.20.202.254:45617), 821e81a3b63c423d84ff6c8d52e540b4 (127.20.202.252:33431)
I20260806 07:56:03.050503 29720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5e46660ad00497fa221e25ce7f4d004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821e81a3b63c423d84ff6c8d52e540b4" is_pre_election: true
I20260806 07:56:03.051118 29720 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e46660ad00497fa221e25ce7f4d004 in term 0.
I20260806 07:56:03.051879 29619 leader_election.cc:304] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821e81a3b63c423d84ff6c8d52e540b4, b5e46660ad00497fa221e25ce7f4d004; no voters:
I20260806 07:56:03.043467 29600 master_runner.cc:187] Error getting master registration for 127.20.202.254:45617: OK, instance_id { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" instance_seqno: 1786002961659961 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:56:03.054263 29600 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:56:03.054706 29772 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:56:03.054883 29772 raft_consensus.cc:493] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:56:03.055006 29772 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:56:03.058733 29772 raft_consensus.cc:515] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.060076 29720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5e46660ad00497fa221e25ce7f4d004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821e81a3b63c423d84ff6c8d52e540b4"
I20260806 07:56:03.060320 29720 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:56:03.064090 29720 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e46660ad00497fa221e25ce7f4d004 in term 1.
I20260806 07:56:03.064399 29772 leader_election.cc:290] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [CANDIDATE]: Term 1 election: Requested vote from peers 8bbe00a5e948468e8914c0c9208a91c2 (127.20.202.254:45617), 821e81a3b63c423d84ff6c8d52e540b4 (127.20.202.252:33431)
I20260806 07:56:03.064870 29619 leader_election.cc:304] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821e81a3b63c423d84ff6c8d52e540b4, b5e46660ad00497fa221e25ce7f4d004; no voters:
I20260806 07:56:03.065244 29772 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:56:03.066490 29772 raft_consensus.cc:697] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 1 LEADER]: Becoming Leader. State: Replica: b5e46660ad00497fa221e25ce7f4d004, State: Running, Role: LEADER
I20260806 07:56:03.067098 29772 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.071626 29785 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e46660ad00497fa221e25ce7f4d004. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.072005 29785 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:03.075618 29787 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:56:03.075744 29591 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } attempt: 5
I20260806 07:56:03.078100 29787 catalog_manager.cc:1520] Initializing Kudu cluster ID...
Aug 06 07:56:03 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002961, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:03.096927 29720 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 1 FOLLOWER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:56:03.098059 29772 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:56:03.121392 29591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Bootstrap starting.
I20260806 07:56:03.129215 29758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e46660ad00497fa221e25ce7f4d004. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.129496 29758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:03.130281 29591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Neither blocks nor log segments found. Creating new log.
W20260806 07:56:03.131435 29699 tablet.cc:2406] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:56:03.133513 29591 log.cc:826] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:03.143471 29591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: No bootstrap required, opened a new log
I20260806 07:56:03.146149 29789 mvcc.cc:204] Tried to move back new op lower bound from 7315468136829501440 to 7315468136728494080. Current Snapshot: MvccSnapshot[applied={T|T < 7315468136829501440}]
I20260806 07:56:03.157071 29591 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.157655 29591 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:03.157850 29591 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbe00a5e948468e8914c0c9208a91c2, State: Initialized, Role: FOLLOWER
I20260806 07:56:03.158812 29591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } }
I20260806 07:56:03.166249 29758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.166479 29758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:03.168429 29591 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:56:03.168502 29564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5e46660ad00497fa221e25ce7f4d004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbe00a5e948468e8914c0c9208a91c2"
I20260806 07:56:03.169006 29564 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:56:03.166268 29794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.170506 29794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:03.172709 29564 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e46660ad00497fa221e25ce7f4d004 in term 1.
I20260806 07:56:03.173121 29565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5e46660ad00497fa221e25ce7f4d004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbe00a5e948468e8914c0c9208a91c2" is_pre_election: true
I20260806 07:56:03.173441 29565 raft_consensus.cc:2376] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b5e46660ad00497fa221e25ce7f4d004 in term 1. Re-sending same reply.
I20260806 07:56:03.174284 29563 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:56:03.174789 29787 catalog_manager.cc:1383] Generated new cluster ID: 55b2c9e1d68a42feac050bd5baaed853
I20260806 07:56:03.174916 29787 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:56:03.182032 29772 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:56:03.197208 29772 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.197649 29772 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:03.238444 29787 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:56:03.241490 29787 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:56:03.251772 29794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e46660ad00497fa221e25ce7f4d004. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.252170 29794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:03.251796 29806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.252398 29806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:03.260455 29772 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } }
I20260806 07:56:03.260826 29772 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:03.273200 29809 catalog_manager.cc:1295] Loaded cluster ID: 55b2c9e1d68a42feac050bd5baaed853
I20260806 07:56:03.273308 29809 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: loading cluster ID for follower catalog manager: success
I20260806 07:56:03.277312 29809 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: acquiring CA information for follower catalog manager: success
W20260806 07:56:03.289310 29809 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 07:56:03.294356 29787 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: Generated new TSK 0
I20260806 07:56:03.295452 29787 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:03.506204 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:03.961180 29816 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:03.961530 29816 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:03.961695 29816 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:03.970136 29816 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:03.970376 29816 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:03.981197 29816 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:03.983086 29816 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:03.984537 29816 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:03.994910 29822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:03.994860 29821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:03.995916 29825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:03.999379 29816 server_base.cc:1061] running on GCE node
I20260806 07:56:03.999969 29816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:04.001134 29816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:04.002440 29816 hybrid_clock.cc:648] HybridClock initialized: now 1786002964002250 us; error 146 us; skew 500 ppm
I20260806 07:56:04.002797 29816 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:04.006506 29816 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:04.008273 29816 webserver.cc:533] Webserver started at http://127.20.202.193:42621/ using document root <none> and password file <none>
I20260806 07:56:04.009105 29816 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:04.009243 29816 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:04.009661 29816 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:04.011299 29769 catalog_manager.cc:1295] Loaded cluster ID: 55b2c9e1d68a42feac050bd5baaed853
I20260806 07:56:04.011394 29769 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: loading cluster ID for follower catalog manager: success
I20260806 07:56:04.013118 29816 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:04.014183 29816 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:04.015157 29769 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: acquiring CA information for follower catalog manager: success
I20260806 07:56:04.017732 29769 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:56:04.020779 29816 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 07:56:04.024533 29832 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:04.026068 29816 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:04.026254 29816 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:04.026443 29816 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:04.042264 29816 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:04.046173 29816 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:04.046479 29816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:04.047503 29816 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:04.049182 29816 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:04.049299 29816 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:04.049461 29816 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:04.049525 29816 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:04.086803 29816 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40587
I20260806 07:56:04.086897 29946 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40587 every 8 connection(s)
I20260806 07:56:04.088322 29816 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:04.090658 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29816
I20260806 07:56:04.090878 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:04.123608 29949 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34107
I20260806 07:56:04.123649 29948 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45617
I20260806 07:56:04.124022 29947 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33431
I20260806 07:56:04.124126 29948 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:04.124166 29947 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:04.125397 29947 heartbeater.cc:507] Master 127.20.202.252:33431 requested a full tablet report, sending...
I20260806 07:56:04.125435 29948 heartbeater.cc:507] Master 127.20.202.254:45617 requested a full tablet report, sending...
I20260806 07:56:04.125435 29949 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:04.126204 29949 heartbeater.cc:507] Master 127.20.202.253:34107 requested a full tablet report, sending...
I20260806 07:56:04.127861 29633 ts_manager.cc:194] Registered new tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:04.128201 29710 ts_manager.cc:194] Registered new tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:04.128183 29555 ts_manager.cc:194] Registered new tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:04.129866 29633 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:60403
I20260806 07:56:04.142817 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:04.274430 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:37231
--webserver_interface=127.20.202.251
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:04.292098 29809 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:56:04.628423 29961 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:04.628738 29961 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:04.628912 29961 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:04.638089 29961 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:04.638192 29961 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:04.638322 29961 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:04.638393 29961 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:04.649356 29961 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:37231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:04.651110 29961 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:04.652544 29961 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:04.661298 29966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:04.661373 29967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:04.662555 29969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:04.663868 29961 server_base.cc:1061] running on GCE node
I20260806 07:56:04.664721 29961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:04.666744 29961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:04.668076 29961 hybrid_clock.cc:648] HybridClock initialized: now 1786002964667888 us; error 151 us; skew 500 ppm
I20260806 07:56:04.668466 29961 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:04.672294 29961 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:56:04.674206 29961 webserver.cc:533] Webserver started at http://127.20.202.251:41587/ using document root <none> and password file <none>
I20260806 07:56:04.674962 29961 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:04.675112 29961 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:04.675457 29961 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:04.678148 29961 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "003c5c667ce54ee79fef9d9eaabdb698"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:04.678992 29961 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "003c5c667ce54ee79fef9d9eaabdb698"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:04.684872 29961 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:56:04.688477 29976 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:04.690084 29961 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:56:04.690292 29961 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "003c5c667ce54ee79fef9d9eaabdb698"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:04.690490 29961 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:04.738298 29961 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:04.742697 29961 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:04.743009 29961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:04.764568 29961 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:37231
I20260806 07:56:04.764600 30027 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:37231 every 8 connection(s)
I20260806 07:56:04.766214 29961 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:56:04.768039 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 29961
I20260806 07:56:04.771667 30028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:04.777547 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:04.796061 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:04.809847 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:04.823917 30028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002964, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:04.842756 30028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Bootstrap starting.
I20260806 07:56:04.846599 30028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:04.847826 30028 log.cc:826] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:04.850818 30028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: No bootstrap required, opened a new log
I20260806 07:56:04.856505 30028 raft_consensus.cc:359] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } }
I20260806 07:56:04.856854 30028 raft_consensus.cc:385] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:04.856976 30028 raft_consensus.cc:740] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003c5c667ce54ee79fef9d9eaabdb698, State: Initialized, Role: FOLLOWER
I20260806 07:56:04.857793 30028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } }
I20260806 07:56:04.858994 30037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } } }
I20260806 07:56:04.859419 30037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:04.859967 30028 sys_catalog.cc:565] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:56:04.870064 30049 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:04.870218 30049 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:04.878207 29961 master_runner.cc:429] Detected that this master 003c5c667ce54ee79fef9d9eaabdb698 is joining an existing cluster
I20260806 07:56:04.878392 29961 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:56:04.879405 29961 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:56:04.879769 29961 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:56:04.879966 29961 tablet_replica.cc:333] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: stopping tablet replica
I20260806 07:56:04.881688 29961 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:37231
Aug 06 07:56:04 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002962, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:04.937148 29633 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:37231: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 37231
}
attrs {
promote: true
}
}
dest_uuid: "b5e46660ad00497fa221e25ce7f4d004"
cas_config_opid_index: -1
I20260806 07:56:04.938920 29633 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } }
I20260806 07:56:04.942246 29563 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:56:04.943063 29618 consensus_peers.cc:597] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 -> Peer 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251:37231): Couldn't send request to peer 003c5c667ce54ee79fef9d9eaabdb698. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:56:04.943063 29811 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:56:04.946928 29719 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 1 FOLLOWER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:56:04.946403 30063 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.947793 29811 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:56:04.948477 29563 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.950892 30063 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:37231
I20260806 07:56:04.951686 29961 master.cc:562] Master@127.20.202.251:37231 shutting down...
I20260806 07:56:04.953939 29719 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.965715 29811 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.966069 29811 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:04.966758 29811 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.964905 30067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.966966 29811 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:04.967068 30067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:04.967566 30066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:04.967963 30066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:04.994536 29961 master.cc:584] Master@127.20.202.251:37231 shutdown complete.
I20260806 07:56:04.994634 29961 master_runner.cc:306] Clearing existing system tablet
I20260806 07:56:04.995123 29961 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:04.995270 29961 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:04.997802 29961 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 07:56:04.999821 30072 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:05.000174 29961 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:05.000454 29961 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 07:56:05.000550 29961 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "003c5c667ce54ee79fef9d9eaabdb698"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:05.005280 29961 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:56:05.012559 29961 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:56:05.012751 29961 log.cc:1199] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:56:05.013242 29961 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Deleting consensus metadata
I20260806 07:56:05.013504 29961 master_runner.cc:316] Copying system tablet from 127.20.202.253:34107
I20260806 07:56:05.016395 29961 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.253:34107
I20260806 07:56:05.031155 29653 tablet_copy_service.cc:140] P b5e46660ad00497fa221e25ce7f4d004: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 003c5c667ce54ee79fef9d9eaabdb698 ({username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:52426)
I20260806 07:56:05.031477 29653 tablet_copy_service.cc:161] P b5e46660ad00497fa221e25ce7f4d004: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 003c5c667ce54ee79fef9d9eaabdb698 at {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:52426: session id = 003c5c667ce54ee79fef9d9eaabdb698-00000000000000000000000000000000
I20260806 07:56:05.034479 29653 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:56:05.036998 29961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:56:05.044404 29961 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: tablet copy: Starting download of 0 data blocks...
I20260806 07:56:05.044724 29961 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: tablet copy: Starting download of 1 WAL segments...
I20260806 07:56:05.047788 29961 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:56:05.052551 29653 tablet_copy_service.cc:342] P b5e46660ad00497fa221e25ce7f4d004: Request end of tablet copy session 003c5c667ce54ee79fef9d9eaabdb698-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:52426
I20260806 07:56:05.052814 29653 tablet_copy_service.cc:434] P b5e46660ad00497fa221e25ce7f4d004: ending tablet copy session 003c5c667ce54ee79fef9d9eaabdb698-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 003c5c667ce54ee79fef9d9eaabdb698
W20260806 07:56:05.060883 29961 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:56:05.065196 29961 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:05.068027 30079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:05.068126 30080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:05.069567 30082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:05.070477 29961 server_base.cc:1061] running on GCE node
I20260806 07:56:05.070794 29961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:05.071655 29961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:05.072883 29961 hybrid_clock.cc:648] HybridClock initialized: now 1786002965072784 us; error 68 us; skew 500 ppm
I20260806 07:56:05.073137 29961 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:05.075347 29961 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:56:05.076622 29961 webserver.cc:533] Webserver started at http://127.20.202.251:45521/ using document root <none> and password file <none>
I20260806 07:56:05.076922 29961 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:05.077039 29961 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:05.079459 29961 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 07:56:05.081287 30090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:05.081787 29961 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260806 07:56:05.081939 29961 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "003c5c667ce54ee79fef9d9eaabdb698"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:05.082152 29961 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:05.092581 29961 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:05.096024 29961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:05.115468 29961 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:37231
I20260806 07:56:05.115506 30141 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:37231 every 8 connection(s)
I20260806 07:56:05.116451 29961 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:56:05.120011 30142 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:56:05.122460 30142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Bootstrap starting.
I20260806 07:56:05.132400 29949 heartbeater.cc:499] Master 127.20.202.253:34107 was elected leader, sending a full tablet report...
I20260806 07:56:05.136783 30142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:56:05.137324 30142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: Bootstrap complete.
I20260806 07:56:05.138329 30142 raft_consensus.cc:359] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } }
I20260806 07:56:05.138562 30142 raft_consensus.cc:740] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 003c5c667ce54ee79fef9d9eaabdb698, State: Initialized, Role: LEARNER
I20260806 07:56:05.138820 30142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } }
I20260806 07:56:05.139714 30145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:05.139995 30145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:56:05.140036 30142 sys_catalog.cc:565] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:56:05.148751 30156 catalog_manager.cc:1295] Loaded cluster ID: 55b2c9e1d68a42feac050bd5baaed853
I20260806 07:56:05.148842 30156 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: loading cluster ID for follower catalog manager: success
I20260806 07:56:05.151742 30156 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: acquiring CA information for follower catalog manager: success
I20260806 07:56:05.154480 30156 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:56:05.169339 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29816
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:56:05.190042 29942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:05.282555 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29816
I20260806 07:56:05.308904 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40587
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--webserver_port=42621
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:05.451608 30069 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:05.451936 30069 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:05.522853 30117 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:56:05.525856 30145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e46660ad00497fa221e25ce7f4d004. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:05.526165 30145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:56:05.703472 30161 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:05.703826 30161 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:05.704003 30161 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:05.712435 30161 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:05.712702 30161 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:05.725000 30161 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42621
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:05.726758 30161 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:05.728186 30161 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:05.738821 30168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:05.738919 30170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:05.738952 30167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:05.739696 30161 server_base.cc:1061] running on GCE node
I20260806 07:56:05.740300 30161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:05.741662 30161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:05.742964 30161 hybrid_clock.cc:648] HybridClock initialized: now 1786002965742779 us; error 149 us; skew 500 ppm
I20260806 07:56:05.743280 30161 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:05.746837 30161 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:05.748569 30161 webserver.cc:533] Webserver started at http://127.20.202.193:42621/ using document root <none> and password file <none>
I20260806 07:56:05.749344 30161 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:05.749501 30161 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:05.755175 30161 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 07:56:05.758299 30177 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:05.759820 30161 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:05.760015 30161 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:05.760900 30161 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:05.775013 30161 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:05.779660 30161 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:05.780007 30161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:05.781122 30161 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:05.782994 30161 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:05.783128 30161 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:05.783298 30161 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:05.783404 30161 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:05.821544 30161 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40587
I20260806 07:56:05.821666 30291 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40587 every 8 connection(s)
I20260806 07:56:05.823460 30161 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:05.828549 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30161
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:56:05 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:05.877703 30293 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33431
I20260806 07:56:05.878172 30293 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:05.879029 30293 heartbeater.cc:507] Master 127.20.202.252:33431 requested a full tablet report, sending...
I20260806 07:56:05.880200 30294 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45617
I20260806 07:56:05.880344 30294 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:05.880856 30294 heartbeater.cc:507] Master 127.20.202.254:45617 requested a full tablet report, sending...
I20260806 07:56:05.881222 29710 ts_manager.cc:194] Re-registered known tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:05.882292 29555 ts_manager.cc:194] Re-registered known tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:05.883867 30292 heartbeater.cc:344] Connected to a master server at 127.20.202.251:37231
I20260806 07:56:05.883975 30292 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:05.884316 30295 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34107
I20260806 07:56:05.884472 30295 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:05.884516 30292 heartbeater.cc:507] Master 127.20.202.251:37231 requested a full tablet report, sending...
I20260806 07:56:05.884938 30295 heartbeater.cc:507] Master 127.20.202.253:34107 requested a full tablet report, sending...
I20260806 07:56:05.886101 29633 ts_manager.cc:194] Re-registered known tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:05.886227 30107 ts_manager.cc:194] Registered new tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:05.887038 29633 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:53621
I20260806 07:56:05.896827 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:56:05.905659 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:56:05.905840 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29523
I20260806 07:56:05.906853 29585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:05.974539 30069 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:05.974905 30069 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:05.976399 30305 raft_consensus.cc:1064] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004: attempting to promote NON_VOTER 003c5c667ce54ee79fef9d9eaabdb698 to VOTER
I20260806 07:56:05.977234 30305 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } }
I20260806 07:56:05.980247 30117 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 1 LEARNER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:56:05.980386 29719 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 1 FOLLOWER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:56:05.981014 30306 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:56:05.981429 30309 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:56:05.985713 30306 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.986506 30117 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.989357 30306 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.989807 30305 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.989989 30305 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:05.989956 30145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.989557 30306 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:05.990182 30145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003c5c667ce54ee79fef9d9eaabdb698 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:05.989917 30309 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.990614 30309 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:05.990996 29719 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.994094 30312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:05.994388 30312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821e81a3b63c423d84ff6c8d52e540b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:06.034049 29563 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Refusing update from remote peer b5e46660ad00497fa221e25ce7f4d004: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:56:06.034495 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29523
I20260806 07:56:06.035504 30309 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:56:06.059763 29620 connection.cc:573] client connection to 127.20.202.254:45617 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:56:06.060339 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45617
--webserver_interface=127.20.202.254
--webserver_port=41401
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:06.061928 29620 consensus_peers.cc:597] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 -> Peer 8bbe00a5e948468e8914c0c9208a91c2 (127.20.202.254:45617): Couldn't send request to peer 8bbe00a5e948468e8914c0c9208a91c2. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:45617: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:56:06.429363 30315 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:06.429814 30315 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:06.429973 30315 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:06.438799 30315 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:06.438941 30315 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:06.439064 30315 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:06.439154 30315 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:06.450489 30315 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=41401
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:06.452275 30315 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:06.453850 30315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:06.463003 30320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:06.463299 30321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:06.463914 30324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:06.464949 30315 server_base.cc:1061] running on GCE node
I20260806 07:56:06.465811 30315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:06.467460 30315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:06.468741 30315 hybrid_clock.cc:648] HybridClock initialized: now 1786002966468602 us; error 100 us; skew 500 ppm
I20260806 07:56:06.469080 30315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:06 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002966, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:06.472579 30315 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:56:06.474406 30315 webserver.cc:533] Webserver started at http://127.20.202.254:41401/ using document root <none> and password file <none>
I20260806 07:56:06.475111 30315 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:06.475225 30315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:06.480918 30315 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 07:56:06.484006 30331 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:06.485366 30315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:56:06.485556 30315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "8bbe00a5e948468e8914c0c9208a91c2"
format_stamp: "Formatted at 2026-08-06 07:56:01 on dist-test-slave-g350"
I20260806 07:56:06.486315 30315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:06.500432 30315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:06.504356 30315 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:06.504655 30315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:06.525377 30315 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45617
I20260806 07:56:06.525395 30383 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45617 every 8 connection(s)
I20260806 07:56:06.527184 30315 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:56:06.532815 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30315
I20260806 07:56:06.533128 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30161
I20260806 07:56:06.535460 30384 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:56:06.540262 30384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Bootstrap starting.
I20260806 07:56:06.552961 30287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:06.562228 30384 log.cc:826] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:06.571169 30384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:56:06.571727 30384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: Bootstrap complete.
I20260806 07:56:06.578892 30384 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } }
I20260806 07:56:06.580051 30384 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbe00a5e948468e8914c0c9208a91c2, State: Initialized, Role: FOLLOWER
I20260806 07:56:06.580936 30384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } }
I20260806 07:56:06.582080 30392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: true } } }
I20260806 07:56:06.582559 30392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:06.583365 30384 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:56:06.601131 30403 catalog_manager.cc:1295] Loaded cluster ID: 55b2c9e1d68a42feac050bd5baaed853
I20260806 07:56:06.601850 30403 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: loading cluster ID for follower catalog manager: success
I20260806 07:56:06.605880 30403 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: acquiring CA information for follower catalog manager: success
I20260806 07:56:06.608609 30403 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:56:06.612865 30358 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 003c5c667ce54ee79fef9d9eaabdb698 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
Aug 06 07:56:06 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002966, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:06.631078 30392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e46660ad00497fa221e25ce7f4d004. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:06.631322 30392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:06.631739 30408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:06.632129 30408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bbe00a5e948468e8914c0c9208a91c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:06.674988 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30161
I20260806 07:56:06.694907 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40587
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--webserver_port=42621
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:07.058569 30412 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:07.058940 30412 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:07.059105 30412 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:56:07.065977 30309 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e46660ad00497fa221e25ce7f4d004" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bbe00a5e948468e8914c0c9208a91c2" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45617 } } peers { permanent_uuid: "b5e46660ad00497fa221e25ce7f4d004" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 34107 } } peers { permanent_uuid: "821e81a3b63c423d84ff6c8d52e540b4" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 33431 } } peers { permanent_uuid: "003c5c667ce54ee79fef9d9eaabdb698" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37231 } attrs { promote: false } } }
I20260806 07:56:07.066327 30309 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e46660ad00497fa221e25ce7f4d004 [sys.catalog]: This master's current role is: LEADER
W20260806 07:56:07.068612 30412 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:07.068792 30412 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:07.079756 30412 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42621
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:07.081424 30412 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:07.082886 30412 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:07.093744 30420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:07.093693 30421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:07.094350 30423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:07.095384 30412 server_base.cc:1061] running on GCE node
I20260806 07:56:07.096015 30412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:07.097232 30412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:07.100600 30412 hybrid_clock.cc:648] HybridClock initialized: now 1786002967100309 us; error 234 us; skew 500 ppm
I20260806 07:56:07.100952 30412 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:07 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:07.104378 30412 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:07.106107 30412 webserver.cc:533] Webserver started at http://127.20.202.193:42621/ using document root <none> and password file <none>
I20260806 07:56:07.106851 30412 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:07.106995 30412 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:07.112666 30412 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:56:07.115710 30430 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:07.117231 30412 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:56:07.117413 30412 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:07.118209 30412 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:07.144836 30412 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:07.148943 30412 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:07.149242 30412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:07.150399 30412 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:07.152074 30412 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:07.152163 30412 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:07.152288 30412 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:07.152390 30412 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:07.189386 30412 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40587
I20260806 07:56:07.189486 30545 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40587 every 8 connection(s)
I20260806 07:56:07.191084 30412 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:07.200977 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30412
Aug 06 07:56:07 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:56:07 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:07 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:56:07 dist-test-slave-g350 krb5kdc[29499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:07.238835 30546 heartbeater.cc:344] Connected to a master server at 127.20.202.251:37231
I20260806 07:56:07.239303 30546 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:07.240890 30546 heartbeater.cc:507] Master 127.20.202.251:37231 requested a full tablet report, sending...
I20260806 07:56:07.242071 30547 heartbeater.cc:344] Connected to a master server at 127.20.202.252:33431
I20260806 07:56:07.242249 30547 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:07.242844 30547 heartbeater.cc:507] Master 127.20.202.252:33431 requested a full tablet report, sending...
I20260806 07:56:07.243290 30548 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45617
I20260806 07:56:07.243446 30548 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:07.243997 30548 heartbeater.cc:507] Master 127.20.202.254:45617 requested a full tablet report, sending...
I20260806 07:56:07.244521 30107 ts_manager.cc:194] Re-registered known tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:07.244684 29710 ts_manager.cc:194] Re-registered known tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:07.245477 30549 heartbeater.cc:344] Connected to a master server at 127.20.202.253:34107
I20260806 07:56:07.245676 30549 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:07.246220 30549 heartbeater.cc:507] Master 127.20.202.253:34107 requested a full tablet report, sending...
I20260806 07:56:07.246284 30348 ts_manager.cc:194] Registered new tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:07.247335 29633 ts_manager.cc:194] Re-registered known tserver with Master: 893ee44e4cfc4eaa92519793e0d11f34 (127.20.202.193:40587)
I20260806 07:56:07.248567 29633 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:43919
I20260806 07:56:07.281361 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30412
W20260806 07:56:07.282931 30563 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:07.278682 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.20.202.193:40587)
0806 07:56:07.279620 (+ 938us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:07.279628 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:07.279662 (+ 34us) server_negotiation.cc:408] Connection header received
0806 07:56:07.279730 (+ 68us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:07.279737 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:07.279967 (+ 230us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:07.280152 (+ 185us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:07.281287 (+ 1135us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49740: BlockingRecv error: recv got EOF from 127.0.0.1:49740 (error 108)
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":540,"threads_started":1}
W20260806 07:56:07.283891 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30412: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:40587: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:56:07.284015 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30412
I20260806 07:56:07.304188 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40587
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--webserver_port=42621
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:07.652365 30565 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:07.652652 30565 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:07.652742 30565 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:56:07.652802 30565 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:07.661743 30565 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:07.661954 30565 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:07.672339 30565 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42621
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:07.673900 30565 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:07.675221 30565 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:07.685508 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:07.685753 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:07.686931 30573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:07.687335 30565 server_base.cc:1061] running on GCE node
I20260806 07:56:07.688036 30565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:07.689220 30565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:07.692531 30565 hybrid_clock.cc:648] HybridClock initialized: now 1786002967692327 us; error 158 us; skew 500 ppm
I20260806 07:56:07.692862 30565 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:07 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002967, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:07.696580 30565 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:07.698501 30565 webserver.cc:533] Webserver started at http://127.20.202.193:42621/ using document root <none> and password file <none>
I20260806 07:56:07.699316 30565 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:07.699407 30565 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:07.705680 30565 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 07:56:07.708703 30580 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:07.710354 30565 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:56:07.710559 30565 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:07.711398 30565 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:07.747149 30565 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:07.751770 30565 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:07.752318 30565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:07.753433 30565 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:07.755281 30565 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:07.755367 30565 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:07.755554 30565 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:07.755620 30565 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:56:07.756196 30565 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:56:07.756278 30565 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:56:07.756385 30565 fault_injection.cc:59] injecting fault for kudu (pid 30565): FLAGS_fault_before_start (process will exit)
I20260806 07:56:07.777285 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:40587
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431
--webserver_port=42621
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:08.146642 30588 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:08.147034 30588 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:08.147228 30588 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:08.156239 30588 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:08.156533 30588 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:08.167574 30588 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35581,127.20.202.213:35791,127.20.202.214:40499,127.20.202.215:42881,127.20.202.216:42757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:40587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=42621
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45617,127.20.202.253:34107,127.20.202.252:33431,127.20.202.251:37231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:08.169307 30588 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:08.170775 30588 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:08.180895 30595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:08.180894 30596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:08.182366 30598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:08.182775 30588 server_base.cc:1061] running on GCE node
I20260806 07:56:08.183516 30588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:08.184883 30588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:08.186281 30588 hybrid_clock.cc:648] HybridClock initialized: now 1786002968186125 us; error 125 us; skew 500 ppm
I20260806 07:56:08.186616 30588 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:08 dist-test-slave-g350 krb5kdc[29499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:08.190333 30588 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:08.192404 30588 webserver.cc:533] Webserver started at http://127.20.202.193:42621/ using document root <none> and password file <none>
I20260806 07:56:08.193224 30588 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:08.193387 30588 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:08.199745 30588 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:56:08.202912 30605 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:08.204617 30588 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:56:08.204754 30588 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "893ee44e4cfc4eaa92519793e0d11f34"
format_stamp: "Formatted at 2026-08-06 07:56:04 on dist-test-slave-g350"
I20260806 07:56:08.205492 30588 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:08.221352 30588 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:08.230700 30588 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:08.230955 30588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:08.231925 30588 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:08.233572 30588 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:08.233726 30588 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:56:08.233811 30588 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:08.233867 30588 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:08.270659 30588 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40587
I20260806 07:56:08.270767 30720 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40587 every 8 connection(s)
I20260806 07:56:08.272287 30588 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:08.281502 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 30588
I20260806 07:56:08.281913 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30588
W20260806 07:56:08.304184 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30588: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:40587: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:56:08.304330 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30588
I20260806 07:56:08.322960 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30315
W20260806 07:56:08.323653 30590 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:08.295383 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:60237 (local address 127.20.202.254:45617)
0806 07:56:08.295621 (+ 238us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:08.295630 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:08.299231 (+ 3601us) server_negotiation.cc:408] Connection header received
0806 07:56:08.299645 (+ 414us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:08.299653 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:08.299741 (+ 88us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:08.299848 (+ 107us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:08.322674 (+ 22826us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:60237: BlockingRecv error: recv got EOF from 127.20.202.193:60237 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 07:56:08.323709 30727 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:08.277134 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:46987 (local address 127.20.202.251:37231)
0806 07:56:08.283169 (+ 6035us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:08.283176 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:08.295280 (+ 12104us) server_negotiation.cc:408] Connection header received
0806 07:56:08.297904 (+ 2624us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:08.297912 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:08.298001 (+ 89us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:08.298103 (+ 102us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:08.303175 (+ 5072us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:56:08.304624 (+ 1449us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:56:08.322736 (+ 18112us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:46987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":5874,"thread_start_us":6199,"threads_started":1}
W20260806 07:56:08.323709 30729 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:08.277877 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:45615 (local address 127.20.202.253:34107)
0806 07:56:08.284200 (+ 6323us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:08.284208 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:08.297004 (+ 12796us) server_negotiation.cc:408] Connection header received
0806 07:56:08.298565 (+ 1561us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:08.298572 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:08.298654 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:08.298752 (+ 98us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:08.303858 (+ 5106us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:56:08.305865 (+ 2007us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:56:08.322736 (+ 16871us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:45615: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":6187,"thread_start_us":6071,"threads_started":1}
W20260806 07:56:08.325232 30730 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:08.278221 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:54939 (local address 127.20.202.252:33431)
0806 07:56:08.284613 (+ 6392us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:08.284621 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:08.297711 (+ 13090us) server_negotiation.cc:408] Connection header received
0806 07:56:08.297914 (+ 203us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:08.297919 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:08.298001 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:08.298198 (+ 197us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:08.303470 (+ 5272us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:56:08.306142 (+ 2672us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:56:08.324332 (+ 18190us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:54939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":6256,"thread_start_us":6122,"threads_started":1}
W20260806 07:56:08.326066 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 30315: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:45617: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:56:08.326231 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 30315
I20260806 07:56:08.350633 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29600
I20260806 07:56:08.352015 29663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:08.470571 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29600
I20260806 07:56:08.495627 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29678
I20260806 07:56:08.497097 29741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:08.609929 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29678
I20260806 07:56:08.633260 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 29961
I20260806 07:56:08.634567 30137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:08.750985 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 29961
2026-08-06T07:56:08Z chronyd exiting
2026-08-06T07:56:08Z chronyd exiting
2026-08-06T07:56:08Z chronyd exiting
2026-08-06T07:56:08Z chronyd exiting
2026-08-06T07:56:08Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/8 (10319 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/9
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:56:08 dist-test-slave-g350 krb5kdc[30740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:56:08 dist-test-slave-g350 krb5kdc[30740](info): set up 2 sockets
Aug 06 07:56:08 dist-test-slave-g350 krb5kdc[30740](info): commencing operation
krb5kdc: starting...
W20260806 07:56:10.917707 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.077s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:56:11 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:56:18: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:56:20 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002980, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:56:27 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002987, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:56:27 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786002987, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:29.457531 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40765
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:29.817762 31042 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:29.818107 31042 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:29.818195 31042 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:29.826742 31042 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:29.826853 31042 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:29.826910 31042 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:29.826951 31042 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:29.838138 31042 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:29.839808 31042 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:29.841178 31042 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:29.850019 31048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:29.849993 31047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:29.850172 31050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:29.851053 31042 server_base.cc:1061] running on GCE node
I20260806 07:56:29.851850 31042 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:29.852025 31042 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:29.852147 31042 hybrid_clock.cc:648] HybridClock initialized: now 1786002989852109 us; error 0 us; skew 500 ppm
I20260806 07:56:29.852509 31042 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:29 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:29.856091 31042 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:56:29.858237 31042 webserver.cc:533] Webserver started at http://127.20.202.254:45225/ using document root <none> and password file <none>
I20260806 07:56:29.858997 31042 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:29.859143 31042 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:29.859503 31042 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:29.862043 31042 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "229e6c940d4f4afbbfb5ca41248c0b57"
format_stamp: "Formatted at 2026-08-06 07:56:29 on dist-test-slave-g350"
I20260806 07:56:29.862857 31042 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "229e6c940d4f4afbbfb5ca41248c0b57"
format_stamp: "Formatted at 2026-08-06 07:56:29 on dist-test-slave-g350"
I20260806 07:56:29.868570 31042 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:56:29.872558 31056 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:29.874184 31042 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:29.874365 31042 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "229e6c940d4f4afbbfb5ca41248c0b57"
format_stamp: "Formatted at 2026-08-06 07:56:29 on dist-test-slave-g350"
I20260806 07:56:29.874547 31042 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:29.894402 31042 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:29.899005 31042 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:29.899322 31042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:29.920699 31042 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40765
I20260806 07:56:29.920719 31108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40765 every 8 connection(s)
I20260806 07:56:29.922361 31042 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:56:29.927047 31109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:29.931989 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31042
I20260806 07:56:29.932200 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:56:29.934374 31109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:29 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:29.961253 31109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:29.962853 31059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38367: connect: Connection refused (error 111)
W20260806 07:56:29.964742 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38367: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38367: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:30.014166 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } attempt: 1
W20260806 07:56:30.016463 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38367: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38367: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:30.077517 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38367
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:30.084797 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } attempt: 2
W20260806 07:56:30.087091 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38367: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38367: connect: Connection refused (error 111)
I20260806 07:56:30.178421 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } attempt: 3
W20260806 07:56:30.180980 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38367: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38367: connect: Connection refused (error 111)
I20260806 07:56:30.324299 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } attempt: 4
W20260806 07:56:30.326835 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38367: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38367: connect: Connection refused (error 111)
W20260806 07:56:30.433112 31118 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:30.433547 31118 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:30.433737 31118 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:30.442394 31118 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:30.442564 31118 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:30.442695 31118 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:30.442792 31118 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:30.453881 31118 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:30.455740 31118 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:30.457195 31118 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:30.466190 31124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:30.466223 31123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:30.467195 31118 server_base.cc:1061] running on GCE node
W20260806 07:56:30.467279 31126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:30.468063 31118 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:30.468261 31118 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:30.468336 31118 hybrid_clock.cc:648] HybridClock initialized: now 1786002990468317 us; error 0 us; skew 500 ppm
I20260806 07:56:30.468657 31118 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:30 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002990, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:30.472427 31118 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:56:30.474591 31118 webserver.cc:533] Webserver started at http://127.20.202.253:43439/ using document root <none> and password file <none>
I20260806 07:56:30.475363 31118 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:30.475486 31118 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:30.475867 31118 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:30.478430 31118 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "5bb4d564605a439aa8b2ef5f84e428ba"
format_stamp: "Formatted at 2026-08-06 07:56:30 on dist-test-slave-g350"
I20260806 07:56:30.479414 31118 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "5bb4d564605a439aa8b2ef5f84e428ba"
format_stamp: "Formatted at 2026-08-06 07:56:30 on dist-test-slave-g350"
I20260806 07:56:30.485071 31118 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:56:30.488783 31132 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:30.490250 31118 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:30.490450 31118 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "5bb4d564605a439aa8b2ef5f84e428ba"
format_stamp: "Formatted at 2026-08-06 07:56:30 on dist-test-slave-g350"
I20260806 07:56:30.490672 31118 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:30.504077 31118 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:30.508134 31118 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:30.508402 31118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:30.530172 31118 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:38367
I20260806 07:56:30.530217 31184 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:38367 every 8 connection(s)
I20260806 07:56:30.531800 31118 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:56:30.536372 31185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:30.541335 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31118
I20260806 07:56:30.541585 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:56:30.543018 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:30 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002990, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:30.564580 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:56:30 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002990, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:30.583885 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:30.585253 31136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
W20260806 07:56:30.587131 31185 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:30.626111 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } attempt: 5
Aug 06 07:56:30 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:30.636507 31185 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 1
W20260806 07:56:30.638564 31185 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:56:30.646194 31109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:30.648346 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:30.681795 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:40117
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:30.699609 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 1
W20260806 07:56:30.701848 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
I20260806 07:56:30.706799 31185 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 2
W20260806 07:56:30.708958 31185 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
I20260806 07:56:30.770170 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 2
W20260806 07:56:30.772539 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
I20260806 07:56:30.800259 31185 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 3
W20260806 07:56:30.802625 31185 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
I20260806 07:56:30.878823 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 3
W20260806 07:56:30.881269 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
I20260806 07:56:30.945906 31185 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 4
W20260806 07:56:30.948283 31185 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
W20260806 07:56:31.041307 31195 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:31.041704 31195 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:31.041901 31195 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:31.050438 31195 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:31.050559 31195 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:31.050643 31195 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:31.050721 31195 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:31.058571 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 4
W20260806 07:56:31.060789 31109 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:40117: Network error: Client connection negotiation failed: client connection to 127.20.202.252:40117: connect: Connection refused (error 111)
I20260806 07:56:31.062019 31195 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:40117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:31.063671 31195 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:31.064985 31195 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:31.073348 31201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:31.074294 31200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:31.078896 31203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:31.079350 31195 server_base.cc:1061] running on GCE node
I20260806 07:56:31.080295 31195 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:31.080531 31195 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:31.080628 31195 hybrid_clock.cc:648] HybridClock initialized: now 1786002991080609 us; error 0 us; skew 500 ppm
I20260806 07:56:31.080993 31195 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002991, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:31.084589 31195 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:56:31.086854 31195 webserver.cc:533] Webserver started at http://127.20.202.252:33295/ using document root <none> and password file <none>
I20260806 07:56:31.087651 31195 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:31.087806 31195 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:31.088166 31195 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:31.090785 31195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "0fdbf57042934e059ea4dc600434260a"
format_stamp: "Formatted at 2026-08-06 07:56:31 on dist-test-slave-g350"
I20260806 07:56:31.091621 31195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "0fdbf57042934e059ea4dc600434260a"
format_stamp: "Formatted at 2026-08-06 07:56:31 on dist-test-slave-g350"
I20260806 07:56:31.097352 31195 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 07:56:31.101214 31209 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:31.102900 31195 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:31.103124 31195 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "0fdbf57042934e059ea4dc600434260a"
format_stamp: "Formatted at 2026-08-06 07:56:31 on dist-test-slave-g350"
I20260806 07:56:31.103340 31195 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:31.130775 31195 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:31.135111 31195 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:31.135414 31195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:31.156476 31195 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:40117
I20260806 07:56:31.156497 31260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:40117 every 8 connection(s)
I20260806 07:56:31.158159 31195 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:56:31.162485 31261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:31.164701 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31195
I20260806 07:56:31.164909 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:56:31.171550 31261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002991, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:31.194257 31261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002991, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:31.209106 31261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002991, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:31.232643 31261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: Bootstrap starting.
I20260806 07:56:31.236217 31261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:31.237306 31261 log.cc:826] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:31.240034 31261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: No bootstrap required, opened a new log
I20260806 07:56:31.245115 31261 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.245434 31261 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:31.245584 31261 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fdbf57042934e059ea4dc600434260a, State: Initialized, Role: FOLLOWER
I20260806 07:56:31.246230 31261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.247295 31272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.247565 31185 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 5
I20260806 07:56:31.247658 31272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:31.248636 31261 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002990, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:31.257689 31195 master_runner.cc:187] Error getting master registration for 127.20.202.254:40765: OK, instance_id { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" instance_seqno: 1786002989914940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:56:31.264611 31195 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:56:31.270435 31283 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:31.270697 31283 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:31.273638 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: Bootstrap starting.
I20260806 07:56:31.277032 31185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:31.278293 31185 log.cc:826] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:31.281188 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: No bootstrap required, opened a new log
I20260806 07:56:31.287907 31185 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.288225 31185 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:31.288388 31185 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb4d564605a439aa8b2ef5f84e428ba, State: Initialized, Role: FOLLOWER
I20260806 07:56:31.289090 31185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.290180 31286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.290501 31286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:31.291365 31185 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:56:31.306062 31297 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:31.306310 31297 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:31.306867 31272 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:56:31.307147 31272 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.309027 31272 leader_election.cc:290] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 229e6c940d4f4afbbfb5ca41248c0b57 (127.20.202.254:40765), 5bb4d564605a439aa8b2ef5f84e428ba (127.20.202.253:38367)
I20260806 07:56:31.302575 31118 master_runner.cc:187] Error getting master registration for 127.20.202.254:40765: OK, instance_id { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" instance_seqno: 1786002989914940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:56:31.313104 31118 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:56:31.313408 31160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fdbf57042934e059ea4dc600434260a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" is_pre_election: true
I20260806 07:56:31.314147 31160 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fdbf57042934e059ea4dc600434260a in term 0.
I20260806 07:56:31.314895 31212 leader_election.cc:304] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fdbf57042934e059ea4dc600434260a, 5bb4d564605a439aa8b2ef5f84e428ba; no voters:
I20260806 07:56:31.315438 31272 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:56:31.315627 31272 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:56:31.315771 31272 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:56:31.319756 31272 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.320700 31272 leader_election.cc:290] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [CANDIDATE]: Term 1 election: Requested vote from peers 229e6c940d4f4afbbfb5ca41248c0b57 (127.20.202.254:40765), 5bb4d564605a439aa8b2ef5f84e428ba (127.20.202.253:38367)
I20260806 07:56:31.321686 31160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fdbf57042934e059ea4dc600434260a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bb4d564605a439aa8b2ef5f84e428ba"
I20260806 07:56:31.321914 31160 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:56:31.326045 31160 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fdbf57042934e059ea4dc600434260a in term 1.
I20260806 07:56:31.328456 31212 leader_election.cc:304] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fdbf57042934e059ea4dc600434260a, 5bb4d564605a439aa8b2ef5f84e428ba; no voters:
I20260806 07:56:31.329140 31272 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:56:31.330207 31272 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 1 LEADER]: Becoming Leader. State: Replica: 0fdbf57042934e059ea4dc600434260a, State: Running, Role: LEADER
I20260806 07:56:31.331568 31272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.338023 31109 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } attempt: 5
I20260806 07:56:31.338461 31299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fdbf57042934e059ea4dc600434260a. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.338812 31299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
Aug 06 07:56:31 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:31.350378 31302 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:56:31.352975 31302 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:56:31.364220 31109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Bootstrap starting.
I20260806 07:56:31.368801 31160 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 1 FOLLOWER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:56:31.370419 31299 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:56:31.371198 31109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:31.373373 31109 log.cc:826] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Log is configured to *not* fsync() on all Append() calls
W20260806 07:56:31.387629 31215 tablet.cc:2406] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:56:31.389073 31109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: No bootstrap required, opened a new log
I20260806 07:56:31.397779 31109 raft_consensus.cc:359] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.398289 31109 raft_consensus.cc:385] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:31.398465 31109 raft_consensus.cc:740] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229e6c940d4f4afbbfb5ca41248c0b57, State: Initialized, Role: FOLLOWER
I20260806 07:56:31.399359 31109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } }
I20260806 07:56:31.400075 31286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fdbf57042934e059ea4dc600434260a. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.400388 31286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:31.407181 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.409292 31303 mvcc.cc:204] Tried to move back new op lower bound from 7315468252635398144 to 7315468252506099712. Current Snapshot: MvccSnapshot[applied={T|T < 7315468252635398144}]
I20260806 07:56:31.410126 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:31.410027 31084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fdbf57042934e059ea4dc600434260a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" is_pre_election: true
I20260806 07:56:31.409926 31109 sys_catalog.cc:565] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:56:31.410714 31084 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fdbf57042934e059ea4dc600434260a in term 0.
I20260806 07:56:31.411149 31083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fdbf57042934e059ea4dc600434260a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229e6c940d4f4afbbfb5ca41248c0b57"
I20260806 07:56:31.411377 31082 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:56:31.415947 31082 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:56:31.417114 31299 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:56:31.420390 31302 catalog_manager.cc:1383] Generated new cluster ID: e0e84f5ebc204dc7ae07e3ad189d08ca
I20260806 07:56:31.420517 31302 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:56:31.432812 31299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.433085 31299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:31.434103 31286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.434348 31286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:31.441294 31302 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:56:31.443480 31302 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:56:31.471020 31302 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: Generated new TSK 0
I20260806 07:56:31.472633 31302 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:56:31.487329 31299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.487586 31299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:31.487582 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fdbf57042934e059ea4dc600434260a. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.487911 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:31.488472 31315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } }
I20260806 07:56:31.488828 31315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
I20260806 07:56:31.572901 31327 catalog_manager.cc:1295] Loaded cluster ID: e0e84f5ebc204dc7ae07e3ad189d08ca
I20260806 07:56:31.573043 31327 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: loading cluster ID for follower catalog manager: success
I20260806 07:56:31.577469 31327 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: acquiring CA information for follower catalog manager: success
I20260806 07:56:31.592937 31327 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
W20260806 07:56:31.617345 21291 mini_kdc.cc:268] Time spent creating service keytab for kudu/127.20.202.193: real 0.116s user 0.003s sys 0.031s
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:31.707406 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:32.186651 31332 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:32.186966 31332 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:32.187160 31332 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:32.195808 31332 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:32.196048 31332 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:32.206781 31332 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:32.208864 31332 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:32.210382 31332 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:32.220954 31337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:32.220970 31338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:32.221458 31340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:32.222360 31332 server_base.cc:1061] running on GCE node
I20260806 07:56:32.222937 31332 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:32.223085 31332 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:32.223155 31332 hybrid_clock.cc:648] HybridClock initialized: now 1786002992223134 us; error 0 us; skew 500 ppm
I20260806 07:56:32.223405 31332 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:32 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:32.226923 31332 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:32.228556 31332 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:56:32.229256 31332 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:32.229357 31332 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:32.229753 31332 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:32.232792 31332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:32.233575 31332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:32.239117 31332 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 07:56:32.242497 31346 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:32.243942 31332 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:56:32.244153 31332 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:32.244354 31332 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:32.259483 31332 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:32.263832 31332 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:32.264164 31332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:32.265195 31332 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:32.266898 31332 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:32.267007 31332 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:32.267163 31332 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:32.267241 31332 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:32.303985 31332 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43277
I20260806 07:56:32.304082 31460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43277 every 8 connection(s)
I20260806 07:56:32.305570 31332 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:32.308935 31297 catalog_manager.cc:1295] Loaded cluster ID: e0e84f5ebc204dc7ae07e3ad189d08ca
I20260806 07:56:32.309304 31297 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: loading cluster ID for follower catalog manager: success
I20260806 07:56:32.309448 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31332
I20260806 07:56:32.311600 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:56:32.313277 31297 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: acquiring CA information for follower catalog manager: success
I20260806 07:56:32.316725 31297 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 06 07:56:32 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:32 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:32 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:32.349042 31463 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38367
I20260806 07:56:32.349519 31463 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:32.350721 31463 heartbeater.cc:507] Master 127.20.202.253:38367 requested a full tablet report, sending...
I20260806 07:56:32.352505 31462 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40765
I20260806 07:56:32.352723 31462 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:32.353147 31462 heartbeater.cc:507] Master 127.20.202.254:40765 requested a full tablet report, sending...
I20260806 07:56:32.353752 31149 ts_manager.cc:194] Registered new tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:32.355060 31074 ts_manager.cc:194] Registered new tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:32.356864 31461 heartbeater.cc:344] Connected to a master server at 127.20.202.252:40117
I20260806 07:56:32.357014 31461 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:32.357583 31461 heartbeater.cc:507] Master 127.20.202.252:40117 requested a full tablet report, sending...
I20260806 07:56:32.359483 31226 ts_manager.cc:194] Registered new tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:32.361181 31226 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:54779
I20260806 07:56:32.372763 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:32.510237 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:37047
--webserver_interface=127.20.202.251
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:32.869169 31476 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:32.869573 31476 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:32.869761 31476 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:32.878691 31476 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:32.878846 31476 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:32.878969 31476 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:32.879073 31476 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:32.890411 31476 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:37047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:32.892138 31476 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:32.893747 31476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:32.902097 31482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:32.902213 31481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:32.903751 31484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:32.903812 31476 server_base.cc:1061] running on GCE node
I20260806 07:56:32.904608 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:32.904824 31476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:32.904937 31476 hybrid_clock.cc:648] HybridClock initialized: now 1786002992904915 us; error 0 us; skew 500 ppm
I20260806 07:56:32.905279 31476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:32 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:32.909052 31476 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:56:32.911183 31476 webserver.cc:533] Webserver started at http://127.20.202.251:41619/ using document root <none> and password file <none>
I20260806 07:56:32.911924 31476 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:32.912058 31476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:32.912428 31476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:32.915092 31476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "697ba74852b143ec91e7deef77356dc4"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:32.915961 31476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "697ba74852b143ec91e7deef77356dc4"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:32.921828 31476 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:56:32.925690 31490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:32.927331 31476 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:56:32.927567 31476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "697ba74852b143ec91e7deef77356dc4"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:32.927800 31476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:32.954246 31476 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:32.959082 31476 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:32.959427 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:32.982285 31476 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:37047
I20260806 07:56:32.982303 31542 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:37047 every 8 connection(s)
I20260806 07:56:32.984205 31476 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:56:32.988517 31543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:32.994005 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31476
I20260806 07:56:32.995106 31543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:33.017179 31543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:33.031399 31543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:33.045377 31543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002992, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:33.063102 31543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Bootstrap starting.
I20260806 07:56:33.066421 31543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:33.067554 31543 log.cc:826] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:33.070407 31543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: No bootstrap required, opened a new log
I20260806 07:56:33.075831 31543 raft_consensus.cc:359] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } }
I20260806 07:56:33.076148 31543 raft_consensus.cc:385] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:33.076221 31543 raft_consensus.cc:740] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697ba74852b143ec91e7deef77356dc4, State: Initialized, Role: FOLLOWER
I20260806 07:56:33.076845 31543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } }
I20260806 07:56:33.077868 31552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } } }
I20260806 07:56:33.078214 31552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:33.079075 31543 sys_catalog.cc:565] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:56:33.089572 31563 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:33.089742 31563 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:33.097127 31476 master_runner.cc:429] Detected that this master 697ba74852b143ec91e7deef77356dc4 is joining an existing cluster
I20260806 07:56:33.097288 31476 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:56:33.098306 31476 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:56:33.098685 31476 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:56:33.098906 31476 tablet_replica.cc:333] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: stopping tablet replica
I20260806 07:56:33.101019 31476 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:37047
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002991, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:33.159046 31226 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:37047: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "697ba74852b143ec91e7deef77356dc4"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 37047
}
attrs {
promote: true
}
}
dest_uuid: "0fdbf57042934e059ea4dc600434260a"
cas_config_opid_index: -1
I20260806 07:56:33.160457 31226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } }
I20260806 07:56:33.164623 31083 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:56:33.164840 31212 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a -> Peer 697ba74852b143ec91e7deef77356dc4 (127.20.202.251:37047): Couldn't send request to peer 697ba74852b143ec91e7deef77356dc4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:56:33.165587 31159 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 1 FOLLOWER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:56:33.165738 31272 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:56:33.166649 31272 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:56:33.175350 31577 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.176194 31082 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.182847 31160 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.185920 31577 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:37047
I20260806 07:56:33.186582 31577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.186806 31577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:33.186969 31272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.187273 31272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:33.187593 31272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.187872 31272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:33.188226 31476 master.cc:562] Master@127.20.202.251:37047 shutting down...
I20260806 07:56:33.188481 31579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.189214 31579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:33.194453 31581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.194815 31581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:33.207545 31476 master.cc:584] Master@127.20.202.251:37047 shutdown complete.
I20260806 07:56:33.207667 31476 master_runner.cc:306] Clearing existing system tablet
I20260806 07:56:33.208274 31476 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:33.208401 31476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:33.211776 31476 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 07:56:33.213805 31583 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:33.214145 31476 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:33.214407 31476 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 07:56:33.214493 31476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "697ba74852b143ec91e7deef77356dc4"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:33.217878 31476 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:56:33.222576 31476 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:56:33.222745 31476 log.cc:1199] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:56:33.223186 31476 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Deleting consensus metadata
I20260806 07:56:33.223431 31476 master_runner.cc:316] Copying system tablet from 127.20.202.252:40117
I20260806 07:56:33.226519 31476 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.252:40117
I20260806 07:56:33.241246 31246 tablet_copy_service.cc:140] P 0fdbf57042934e059ea4dc600434260a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 697ba74852b143ec91e7deef77356dc4 ({username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:59662)
I20260806 07:56:33.241500 31246 tablet_copy_service.cc:161] P 0fdbf57042934e059ea4dc600434260a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 697ba74852b143ec91e7deef77356dc4 at {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:59662: session id = 697ba74852b143ec91e7deef77356dc4-00000000000000000000000000000000
I20260806 07:56:33.244729 31246 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:56:33.246907 31476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:56:33.252255 31476 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: tablet copy: Starting download of 0 data blocks...
I20260806 07:56:33.252569 31476 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: tablet copy: Starting download of 1 WAL segments...
I20260806 07:56:33.255597 31476 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:56:33.259104 31246 tablet_copy_service.cc:342] P 0fdbf57042934e059ea4dc600434260a: Request end of tablet copy session 697ba74852b143ec91e7deef77356dc4-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:59662
I20260806 07:56:33.259366 31246 tablet_copy_service.cc:434] P 0fdbf57042934e059ea4dc600434260a: ending tablet copy session 697ba74852b143ec91e7deef77356dc4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 697ba74852b143ec91e7deef77356dc4
I20260806 07:56:33.273350 31476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:33.276335 31590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:33.277015 31591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:33.278244 31593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:33.279191 31476 server_base.cc:1061] running on GCE node
I20260806 07:56:33.279655 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:33.279757 31476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:33.279842 31476 hybrid_clock.cc:648] HybridClock initialized: now 1786002993279800 us; error 0 us; skew 500 ppm
I20260806 07:56:33.280057 31476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:33.282420 31476 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:56:33.289748 31476 webserver.cc:533] Webserver started at http://127.20.202.251:40585/ using document root <none> and password file <none>
I20260806 07:56:33.290125 31476 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:33.290223 31476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:33.292819 31476 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 07:56:33.294808 31599 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:33.295295 31476 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 07:56:33.295435 31476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "697ba74852b143ec91e7deef77356dc4"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:33.295622 31476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:33.311380 31476 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:33.315121 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:33.336654 31476 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:37047
I20260806 07:56:33.336678 31653 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:37047 every 8 connection(s)
I20260806 07:56:33.337908 31476 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:56:33.342093 31654 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:56:33.344699 31654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Bootstrap starting.
I20260806 07:56:33.360002 31654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:56:33.360496 31654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: Bootstrap complete.
I20260806 07:56:33.361430 31654 raft_consensus.cc:359] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } }
I20260806 07:56:33.361704 31654 raft_consensus.cc:740] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 697ba74852b143ec91e7deef77356dc4, State: Initialized, Role: LEARNER
I20260806 07:56:33.361958 31654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } }
I20260806 07:56:33.362811 31657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.363121 31654 sys_catalog.cc:565] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:56:33.363142 31657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:56:33.363708 31461 heartbeater.cc:499] Master 127.20.202.252:40117 was elected leader, sending a full tablet report...
I20260806 07:56:33.371899 31668 catalog_manager.cc:1295] Loaded cluster ID: e0e84f5ebc204dc7ae07e3ad189d08ca
I20260806 07:56:33.372020 31668 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: loading cluster ID for follower catalog manager: success
I20260806 07:56:33.374794 31668 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: acquiring CA information for follower catalog manager: success
I20260806 07:56:33.376772 31668 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:56:33.395040 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31332
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:56:33.414743 31456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:33.522984 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31332
I20260806 07:56:33.546535 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43277
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:33.732224 31627 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:56:33.733006 31657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fdbf57042934e059ea4dc600434260a. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:33.733264 31657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:56:33.927929 31672 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:33.928308 31672 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:33.928488 31672 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:33.937040 31672 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:33.937314 31672 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:33.952061 31672 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:33.954481 31672 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:33.956292 31672 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:33.966233 31678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:33.967162 31679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:33.967940 31681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:33.968691 31672 server_base.cc:1061] running on GCE node
I20260806 07:56:33.969234 31672 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:33.969376 31672 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:33.969444 31672 hybrid_clock.cc:648] HybridClock initialized: now 1786002993969425 us; error 0 us; skew 500 ppm
I20260806 07:56:33.969777 31672 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:33 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:33.973204 31672 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:33.975081 31672 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:56:33.975898 31672 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:33.976055 31672 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:33.982075 31672 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:56:33.985071 31687 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:33.986634 31672 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:33.986804 31672 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:33.987542 31672 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:34.024168 31672 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:34.028321 31672 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:34.028623 31672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:34.029773 31672 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:34.031420 31672 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:34.031545 31672 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:34.031690 31672 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:34.031791 31672 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:34.068586 31672 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43277
I20260806 07:56:34.068686 31803 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43277 every 8 connection(s)
I20260806 07:56:34.070235 31672 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:34.080380 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31672
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:34.105463 31806 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40765
I20260806 07:56:34.105659 31807 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38367
I20260806 07:56:34.105997 31806 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:34.106033 31807 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:34.108036 31807 heartbeater.cc:507] Master 127.20.202.253:38367 requested a full tablet report, sending...
I20260806 07:56:34.108134 31804 heartbeater.cc:344] Connected to a master server at 127.20.202.251:37047
I20260806 07:56:34.108253 31804 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:34.108846 31804 heartbeater.cc:507] Master 127.20.202.251:37047 requested a full tablet report, sending...
I20260806 07:56:34.108023 31806 heartbeater.cc:507] Master 127.20.202.254:40765 requested a full tablet report, sending...
I20260806 07:56:34.110291 31805 heartbeater.cc:344] Connected to a master server at 127.20.202.252:40117
I20260806 07:56:34.110445 31805 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:34.110841 31074 ts_manager.cc:194] Re-registered known tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:34.110867 31805 heartbeater.cc:507] Master 127.20.202.252:40117 requested a full tablet report, sending...
I20260806 07:56:34.111946 31226 ts_manager.cc:194] Re-registered known tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:34.112205 31616 ts_manager.cc:194] Registered new tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:34.112852 31226 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:59213
I20260806 07:56:34.113888 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:56:34.114125 31149 ts_manager.cc:194] Re-registered known tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:34.122123 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:56:34.122311 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31042
I20260806 07:56:34.123181 31104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:34.184252 31577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:34.184563 31577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:34.186213 31818 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a: attempting to promote NON_VOTER 697ba74852b143ec91e7deef77356dc4 to VOTER
I20260806 07:56:34.187074 31818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } }
I20260806 07:56:34.189836 31160 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 1 FOLLOWER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:56:34.189980 31627 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 1 LEARNER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:56:34.190599 31818 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:56:34.190961 31817 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:56:34.195076 31818 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.195951 31627 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.199316 31657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.199563 31657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697ba74852b143ec91e7deef77356dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:34.199853 31577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.200073 31577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:34.199908 31817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.200219 31817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:34.199985 31160 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.200359 31577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.200524 31577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
I20260806 07:56:34.203343 31824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.203744 31824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb4d564605a439aa8b2ef5f84e428ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:34.240928 31082 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Refusing update from remote peer 0fdbf57042934e059ea4dc600434260a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:56:34.241459 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31042
I20260806 07:56:34.242828 31818 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [LEADER]: Connected to new peer: Peer: permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 07:56:34.264596 31213 connection.cc:573] client connection to 127.20.202.254:40765 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 07:56:34.264843 31213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:56:34.265717 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40765
--webserver_interface=127.20.202.254
--webserver_port=45225
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--rpc_reuseport=true
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:34.267326 31213 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a -> Peer 229e6c940d4f4afbbfb5ca41248c0b57 (127.20.202.254:40765): Couldn't send request to peer 229e6c940d4f4afbbfb5ca41248c0b57. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:40765: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260806 07:56:34.631551 31827 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:34.631891 31827 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:34.632047 31827 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:34.640503 31827 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:34.640615 31827 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:34.640676 31827 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:34.640761 31827 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:34.652052 31827 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=45225
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:34.653858 31827 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:34.655361 31827 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:34.664170 31835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:34.664484 31832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:34.664518 31833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:34.665123 31827 server_base.cc:1061] running on GCE node
I20260806 07:56:34.665997 31827 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:34.666185 31827 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:34.666286 31827 hybrid_clock.cc:648] HybridClock initialized: now 1786002994666267 us; error 0 us; skew 500 ppm
I20260806 07:56:34.666643 31827 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:34.670152 31827 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:56:34.672196 31827 webserver.cc:533] Webserver started at http://127.20.202.254:45225/ using document root <none> and password file <none>
I20260806 07:56:34.672955 31827 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:34.673115 31827 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:34.678628 31827 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:56:34.681506 31841 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:34.683143 31827 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:34.683322 31827 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "229e6c940d4f4afbbfb5ca41248c0b57"
format_stamp: "Formatted at 2026-08-06 07:56:29 on dist-test-slave-g350"
I20260806 07:56:34.684075 31827 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:34.718230 31827 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:34.722826 31827 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:34.723145 31827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:34.745450 31892 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40765 every 8 connection(s)
I20260806 07:56:34.745488 31827 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40765
I20260806 07:56:34.747742 31827 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:56:34.750259 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31827
I20260806 07:56:34.750569 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31672
I20260806 07:56:34.757702 31894 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:56:34.763342 31894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Bootstrap starting.
I20260806 07:56:34.785800 31894 log.cc:826] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:34.785982 31799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:34.796978 31894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:56:34.797744 31894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: Bootstrap complete.
I20260806 07:56:34.806752 31894 raft_consensus.cc:359] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } }
I20260806 07:56:34.807760 31894 raft_consensus.cc:740] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229e6c940d4f4afbbfb5ca41248c0b57, State: Initialized, Role: FOLLOWER
I20260806 07:56:34.808596 31894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } }
I20260806 07:56:34.809582 31901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: true } } }
I20260806 07:56:34.809978 31901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:34.810961 31894 sys_catalog.cc:565] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:56:34.816455 31868 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 697ba74852b143ec91e7deef77356dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.823890 31913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.824285 31913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:34.825470 31914 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 07:56:34.825579 31901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fdbf57042934e059ea4dc600434260a. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:34.825805 31901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:56:34 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:34.891525 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31672
I20260806 07:56:34.913198 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43277
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:35.249799 31818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fdbf57042934e059ea4dc600434260a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "229e6c940d4f4afbbfb5ca41248c0b57" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40765 } } peers { permanent_uuid: "5bb4d564605a439aa8b2ef5f84e428ba" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38367 } } peers { permanent_uuid: "0fdbf57042934e059ea4dc600434260a" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 40117 } } peers { permanent_uuid: "697ba74852b143ec91e7deef77356dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 37047 } attrs { promote: false } } }
I20260806 07:56:35.250201 31818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a [sys.catalog]: This master's current role is: LEADER
W20260806 07:56:35.282212 31919 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:35.282542 31919 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:35.282648 31919 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:35.292411 31919 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:35.292683 31919 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:35.304852 31919 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:35.306799 31919 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:35.308411 31919 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:35.319144 31927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:35.319206 31928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:35.320495 31930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:35.321172 31919 server_base.cc:1061] running on GCE node
I20260806 07:56:35.321889 31919 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:35.322098 31919 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:35.322180 31919 hybrid_clock.cc:648] HybridClock initialized: now 1786002995322159 us; error 0 us; skew 500 ppm
I20260806 07:56:35.322466 31919 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:35 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:35.325943 31919 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:35.327726 31919 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:56:35.328435 31919 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:35.328542 31919 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:35.334237 31919 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 07:56:35.337502 31936 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:35.339169 31919 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:56:35.339362 31919 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:35.340243 31919 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:35.361383 31919 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:35.366199 31919 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:35.366513 31919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:35.367771 31919 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:35.369642 31919 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:35.369738 31919 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 07:56:35.369880 31919 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:35.369983 31919 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:35.408365 31919 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43277
I20260806 07:56:35.408406 32050 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43277 every 8 connection(s)
I20260806 07:56:35.410213 31919 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:35.417219 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 31919
Aug 06 07:56:35 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:35 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:35 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:56:35 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:35.452087 32054 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38367
I20260806 07:56:35.452579 32054 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:35.453580 32054 heartbeater.cc:507] Master 127.20.202.253:38367 requested a full tablet report, sending...
I20260806 07:56:35.455215 32053 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40765
I20260806 07:56:35.455364 32053 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:35.455961 32053 heartbeater.cc:507] Master 127.20.202.254:40765 requested a full tablet report, sending...
I20260806 07:56:35.455955 31149 ts_manager.cc:194] Re-registered known tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:35.459326 31858 ts_manager.cc:194] Registered new tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
Aug 06 07:56:35 dist-test-slave-g350 krb5kdc[30740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:56:35.463683 32052 heartbeater.cc:344] Connected to a master server at 127.20.202.252:40117
I20260806 07:56:35.463857 32052 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:35.464566 32052 heartbeater.cc:507] Master 127.20.202.252:40117 requested a full tablet report, sending...
I20260806 07:56:35.465842 31226 ts_manager.cc:194] Re-registered known tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:35.466979 31226 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:39563
I20260806 07:56:35.469723 32051 heartbeater.cc:344] Connected to a master server at 127.20.202.251:37047
I20260806 07:56:35.469863 32051 heartbeater.cc:461] Registering TS with master...
I20260806 07:56:35.470193 32051 heartbeater.cc:507] Master 127.20.202.251:37047 requested a full tablet report, sending...
I20260806 07:56:35.471498 31616 ts_manager.cc:194] Re-registered known tserver with Master: c7ab08f648714109a4c20a648612a203 (127.20.202.193:43277)
I20260806 07:56:35.600296 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31919
W20260806 07:56:35.601490 32069 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:35.598094 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.20.202.193:43277)
0806 07:56:35.598750 (+ 656us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:35.598757 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:35.598790 (+ 33us) server_negotiation.cc:408] Connection header received
0806 07:56:35.598908 (+ 118us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:35.598914 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:35.599145 (+ 231us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:35.599318 (+ 173us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:35.600202 (+ 884us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34466: BlockingRecv error: recv got EOF from 127.0.0.1:34466 (error 108)
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":330,"threads_started":1}
W20260806 07:56:35.603024 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31919: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:43277: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:56:35.603154 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31919
I20260806 07:56:35.622076 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43277
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:35.830922 31914 catalog_manager.cc:1295] Loaded cluster ID: e0e84f5ebc204dc7ae07e3ad189d08ca
I20260806 07:56:35.831072 31914 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: loading cluster ID for follower catalog manager: success
I20260806 07:56:35.835707 31914 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: acquiring CA information for follower catalog manager: success
I20260806 07:56:35.838678 31914 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 229e6c940d4f4afbbfb5ca41248c0b57: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:56:35.998718 32071 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:35.999084 32071 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:35.999253 32071 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:56:35.999361 32071 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:36.007869 32071 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:36.008239 32071 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:36.019557 32071 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:36.021302 32071 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:36.022748 32071 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:36.032866 32077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:36.032902 32078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:36.034845 32081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:36.036058 32071 server_base.cc:1061] running on GCE node
I20260806 07:56:36.036695 32071 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:36.036862 32071 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:36.036969 32071 hybrid_clock.cc:648] HybridClock initialized: now 1786002996036930 us; error 0 us; skew 500 ppm
I20260806 07:56:36.037319 32071 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:36 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002996, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:36.041186 32071 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:36.043123 32071 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:56:36.044029 32071 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:36.044154 32071 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:36.050123 32071 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 07:56:36.053243 32087 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:36.054850 32071 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:36.055045 32071 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:36.055826 32071 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:36.099462 32071 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:36.103940 32071 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:36.104264 32071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:36.105373 32071 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:36.107285 32071 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:36.107385 32071 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:36.107515 32071 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:36.107564 32071 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:56:36.108064 32071 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:56:36.108153 32071 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:56:36.108237 32071 fault_injection.cc:59] injecting fault for kudu (pid 32071): FLAGS_fault_before_start (process will exit)
I20260806 07:56:36.126533 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:43277
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--time_source=system_unsync
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:36.490270 32095 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:36.490615 32095 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:36.490772 32095 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:36.500115 32095 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:36.500427 32095 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:56:36.514446 32095 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--time_source=system_unsync
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:43277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40765,127.20.202.253:38367,127.20.202.252:40117,127.20.202.251:37047
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:36.516307 32095 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:36.517784 32095 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:36.527490 32102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:36.527499 32103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:36.528239 32105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:36.528893 32095 server_base.cc:1061] running on GCE node
I20260806 07:56:36.529434 32095 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 07:56:36.529670 32095 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 07:56:36.529784 32095 hybrid_clock.cc:648] HybridClock initialized: now 1786002996529757 us; error 0 us; skew 500 ppm
I20260806 07:56:36.530107 32095 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:36 dist-test-slave-g350 krb5kdc[30740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002996, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:36.533433 32095 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:56:36.535209 32095 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:56:36.535967 32095 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:36.536093 32095 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:36.541694 32095 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 07:56:36.544443 32111 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:36.545811 32095 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:36.545984 32095 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c7ab08f648714109a4c20a648612a203"
format_stamp: "Formatted at 2026-08-06 07:56:32 on dist-test-slave-g350"
I20260806 07:56:36.546715 32095 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:36.573910 32095 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:36.578205 32095 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:36.578500 32095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:36.579546 32095 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:56:36.581184 32095 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:56:36.581272 32095 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:36.581429 32095 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:56:36.581527 32095 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:36.618691 32095 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:43277
I20260806 07:56:36.618789 32225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:43277 every 8 connection(s)
I20260806 07:56:36.620360 32095 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:56:36.623912 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32095
I20260806 07:56:36.624284 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32095
W20260806 07:56:36.654028 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32095: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:43277: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:56:36.654175 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32095
I20260806 07:56:36.673100 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31827
W20260806 07:56:36.673790 32097 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:36.623887 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:50671 (local address 127.20.202.254:40765)
0806 07:56:36.624105 (+ 218us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:36.624112 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:36.642421 (+ 18309us) server_negotiation.cc:408] Connection header received
0806 07:56:36.642716 (+ 295us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:36.642726 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:36.642833 (+ 107us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:36.642921 (+ 88us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:36.648107 (+ 5186us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:56:36.649496 (+ 1389us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:56:36.672929 (+ 23433us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:50671: BlockingRecv error: recv got EOF from 127.20.202.193:50671 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 07:56:36.673851 32236 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:36.640901 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:37295 (local address 127.20.202.252:40117)
0806 07:56:36.657773 (+ 16872us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:36.657783 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:36.672843 (+ 15060us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:37295: BlockingRecv error: recv got EOF from 127.20.202.193:37295 (error 108)
Metrics: {"server-negotiator.queue_time_us":16729,"thread_start_us":16887,"threads_started":1}
W20260806 07:56:36.674562 32234 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:36.625936 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:45963 (local address 127.20.202.251:37047)
0806 07:56:36.638088 (+ 12152us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:36.638099 (+ 11us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:36.639496 (+ 1397us) server_negotiation.cc:408] Connection header received
0806 07:56:36.641749 (+ 2253us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:36.641759 (+ 10us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:36.641916 (+ 157us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:36.642047 (+ 131us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:36.648057 (+ 6010us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:56:36.649496 (+ 1439us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:56:36.673723 (+ 24227us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:45963: BlockingRecv error: recv got EOF from 127.20.202.193:45963 (error 108)
Metrics: {"server-negotiator.queue_time_us":11984,"thread_start_us":11863,"threads_started":1}
W20260806 07:56:36.675213 32238 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:36.642769 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:46975 (local address 127.20.202.253:38367)
0806 07:56:36.658027 (+ 15258us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:36.658031 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:36.674520 (+ 16489us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:46975: BlockingRecv error: recv got EOF from 127.20.202.193:46975 (error 108)
Metrics: {"server-negotiator.queue_time_us":15158,"thread_start_us":15028,"threads_started":1}
W20260806 07:56:36.675916 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31827: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:40765: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:56:36.675983 32097 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:56:36.674231 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.20.202.254:40765)
0806 07:56:36.674394 (+ 163us) server_negotiation.cc:207] Beginning negotiation
0806 07:56:36.674401 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:56:36.674791 (+ 390us) server_negotiation.cc:408] Connection header received
0806 07:56:36.674867 (+ 76us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:56:36.674873 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:56:36.674976 (+ 103us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:56:36.675096 (+ 120us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:56:36.675825 (+ 729us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48388: BlockingRecv error: recv got EOF from 127.0.0.1:48388 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 07:56:36.676085 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31827
I20260806 07:56:36.698437 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31118
I20260806 07:56:36.699803 31180 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 07:56:36.742450 31213 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0fdbf57042934e059ea4dc600434260a -> Peer 229e6c940d4f4afbbfb5ca41248c0b57 (127.20.202.254:40765): Couldn't send request to peer 229e6c940d4f4afbbfb5ca41248c0b57. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:40765: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:56:36.813889 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31118
I20260806 07:56:36.838014 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31195
I20260806 07:56:36.839354 31256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:36.956414 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31195
I20260806 07:56:36.979354 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 31476
I20260806 07:56:36.980592 31649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:56:37.095420 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 31476
Shutting down hive metastore.
[ OK ] ExternalMiniClusterTest.TestAddMaster/9 (28691 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/10
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:56:37 dist-test-slave-g350 krb5kdc[32251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:56:37 dist-test-slave-g350 krb5kdc[32251](info): set up 2 sockets
Aug 06 07:56:37 dist-test-slave-g350 krb5kdc[32251](info): commencing operation
krb5kdc: starting...
W20260806 07:56:39.597565 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.065s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:56:39 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:56:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:56:39Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:56:47: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:56:49 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786003009, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:56:56 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786003016, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:56:56 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786003016, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:58.046283 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36665
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:56:58.399744 32556 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:58.400077 32556 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:58.400249 32556 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:58.408910 32556 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:58.409016 32556 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:58.409106 32556 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:58.409204 32556 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:58.420461 32556 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:58.422259 32556 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:58.423796 32556 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:58.432420 32561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:58.432458 32562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:58.433537 32564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:58.433905 32556 server_base.cc:1061] running on GCE node
I20260806 07:56:58.434619 32556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:58.436062 32556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:58.437309 32556 hybrid_clock.cc:648] HybridClock initialized: now 1786003018437270 us; error 35 us; skew 500 ppm
I20260806 07:56:58.437726 32556 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:58 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003018, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:58.440855 32556 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:56:58.442541 32556 webserver.cc:533] Webserver started at http://127.20.202.254:32961/ using document root <none> and password file <none>
I20260806 07:56:58.443300 32556 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:58.443441 32556 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:58.443800 32556 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:58.446367 32556 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "a5d63be01ac941788f8cb40b4e97a23d"
format_stamp: "Formatted at 2026-08-06 07:56:58 on dist-test-slave-g350"
I20260806 07:56:58.447186 32556 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "a5d63be01ac941788f8cb40b4e97a23d"
format_stamp: "Formatted at 2026-08-06 07:56:58 on dist-test-slave-g350"
I20260806 07:56:58.452796 32556 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:56:58.456578 32572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:58.458017 32556 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:56:58.458241 32556 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "a5d63be01ac941788f8cb40b4e97a23d"
format_stamp: "Formatted at 2026-08-06 07:56:58 on dist-test-slave-g350"
I20260806 07:56:58.458449 32556 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:58.518770 32556 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:58.523178 32556 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:58.523490 32556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:58.545817 32556 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:36665
I20260806 07:56:58.545836 32623 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:36665 every 8 connection(s)
I20260806 07:56:58.547411 32556 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:56:58.550442 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32556
I20260806 07:56:58.550663 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:56:58.552119 32624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:58.558559 32624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:58 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003018, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
I20260806 07:56:58.584790 32624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:58.586356 32575 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:44487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
W20260806 07:56:58.588361 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:44487: Network error: Client connection negotiation failed: client connection to 127.20.202.253:44487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:58.637765 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } attempt: 1
W20260806 07:56:58.640111 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:44487: Network error: Client connection negotiation failed: client connection to 127.20.202.253:44487: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:58.689319 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:44487
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:58.708408 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } attempt: 2
W20260806 07:56:58.710551 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:44487: Network error: Client connection negotiation failed: client connection to 127.20.202.253:44487: connect: Connection refused (error 111)
I20260806 07:56:58.801810 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } attempt: 3
W20260806 07:56:58.804023 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:44487: Network error: Client connection negotiation failed: client connection to 127.20.202.253:44487: connect: Connection refused (error 111)
I20260806 07:56:58.947321 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } attempt: 4
W20260806 07:56:58.949560 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:44487: Network error: Client connection negotiation failed: client connection to 127.20.202.253:44487: connect: Connection refused (error 111)
W20260806 07:56:59.048410 32634 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:59.048803 32634 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:59.048971 32634 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:59.057780 32634 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:59.057962 32634 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:59.058147 32634 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:59.058244 32634 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:59.069777 32634 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:44487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:56:59.071496 32634 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:59.072971 32634 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:59.081478 32640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:59.081486 32639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:59.082867 32642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:59.083283 32634 server_base.cc:1061] running on GCE node
I20260806 07:56:59.084059 32634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:59.085556 32634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:59.086833 32634 hybrid_clock.cc:648] HybridClock initialized: now 1786003019086764 us; error 71 us; skew 500 ppm
I20260806 07:56:59.087201 32634 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:59.090524 32634 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:56:59.092161 32634 webserver.cc:533] Webserver started at http://127.20.202.253:46659/ using document root <none> and password file <none>
I20260806 07:56:59.092912 32634 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:59.093060 32634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:59.093422 32634 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:59.095938 32634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "671cf1daf5974e4d8b8da0ef1b23b481"
format_stamp: "Formatted at 2026-08-06 07:56:59 on dist-test-slave-g350"
I20260806 07:56:59.096740 32634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "671cf1daf5974e4d8b8da0ef1b23b481"
format_stamp: "Formatted at 2026-08-06 07:56:59 on dist-test-slave-g350"
I20260806 07:56:59.102511 32634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:56:59.106002 32649 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:59.107385 32634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:56:59.107568 32634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "671cf1daf5974e4d8b8da0ef1b23b481"
format_stamp: "Formatted at 2026-08-06 07:56:59 on dist-test-slave-g350"
I20260806 07:56:59.107776 32634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:59.124899 32634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:59.128899 32634 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:59.129207 32634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:59.151060 32634 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:44487
I20260806 07:56:59.151166 32700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:44487 every 8 connection(s)
I20260806 07:56:59.152681 32634 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:56:59.157290 32701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:59.161428 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32634
I20260806 07:56:59.161710 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:56:59.163599 32701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:56:59.187040 32701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:59.207903 32701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:56:59.209175 32653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
W20260806 07:56:59.211019 32701 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:59.248883 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } attempt: 5
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003018, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:59.260454 32701 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 1
W20260806 07:56:59.262712 32701 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.265532 32624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
W20260806 07:56:59.267766 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:56:59.307500 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:34043
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:56:59.319016 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 1
W20260806 07:56:59.321285 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.330956 32701 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 2
W20260806 07:56:59.333261 32701 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.389537 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 2
W20260806 07:56:59.391937 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.424551 32701 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 3
W20260806 07:56:59.426970 32701 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.498164 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 3
W20260806 07:56:59.500753 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.570261 32701 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 4
W20260806 07:56:59.572656 32701 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
W20260806 07:56:59.659708 32712 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:56:59.660032 32712 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:56:59.660117 32712 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:56:59.668483 32712 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:56:59.668589 32712 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:56:59.668648 32712 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:56:59.668695 32712 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:56:59.678232 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 4
I20260806 07:56:59.679814 32712 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:34043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
W20260806 07:56:59.680586 32624 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:34043: Network error: Client connection negotiation failed: client connection to 127.20.202.252:34043: connect: Connection refused (error 111)
I20260806 07:56:59.681557 32712 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:56:59.682974 32712 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:56:59.691340 32717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:59.691459 32718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:56:59.691967 32720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:56:59.692356 32712 server_base.cc:1061] running on GCE node
I20260806 07:56:59.693059 32712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:56:59.694511 32712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:56:59.695767 32712 hybrid_clock.cc:648] HybridClock initialized: now 1786003019695709 us; error 61 us; skew 500 ppm
I20260806 07:56:59.696117 32712 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:56:59.699258 32712 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:56:59.700824 32712 webserver.cc:533] Webserver started at http://127.20.202.252:33563/ using document root <none> and password file <none>
I20260806 07:56:59.701507 32712 fs_manager.cc:362] Metadata directory not provided
I20260806 07:56:59.701659 32712 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:56:59.701984 32712 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:56:59.704303 32712 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "c130e294e4ab48bdaa537108a5c9b223"
format_stamp: "Formatted at 2026-08-06 07:56:59 on dist-test-slave-g350"
I20260806 07:56:59.705032 32712 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "c130e294e4ab48bdaa537108a5c9b223"
format_stamp: "Formatted at 2026-08-06 07:56:59 on dist-test-slave-g350"
I20260806 07:56:59.710377 32712 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:56:59.713888 32727 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:56:59.715247 32712 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:56:59.715375 32712 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "c130e294e4ab48bdaa537108a5c9b223"
format_stamp: "Formatted at 2026-08-06 07:56:59 on dist-test-slave-g350"
I20260806 07:56:59.715533 32712 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:56:59.730871 32712 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:56:59.734899 32712 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:56:59.735145 32712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:56:59.755458 32712 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:34043
I20260806 07:56:59.755488 311 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:34043 every 8 connection(s)
I20260806 07:56:59.757067 32712 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:56:59.760108 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 32712
I20260806 07:56:59.760311 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:56:59.761579 312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:56:59.770229 312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:56:59.792454 312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:59.807704 312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:59.829420 312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: Bootstrap starting.
I20260806 07:56:59.832809 312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:59.833958 312 log.cc:826] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:59.836560 312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: No bootstrap required, opened a new log
I20260806 07:56:59.842080 312 raft_consensus.cc:359] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:56:59.842345 312 raft_consensus.cc:385] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:59.842490 312 raft_consensus.cc:740] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c130e294e4ab48bdaa537108a5c9b223, State: Initialized, Role: FOLLOWER
I20260806 07:56:59.843124 312 consensus_queue.cc:260] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:56:59.843992 322 sys_catalog.cc:455] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:56:59.844269 322 sys_catalog.cc:458] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:59.845090 312 sys_catalog.cc:565] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:56:59.854290 333 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:59.854427 333 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:59.852399 32712 master_runner.cc:187] Error getting master registration for 127.20.202.254:36665: OK, instance_id { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" instance_seqno: 1786003018539877 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:56:59.859182 32712 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:56:59.871920 32701 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 5
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:59.890729 32701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: Bootstrap starting.
I20260806 07:56:59.894587 32701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: Neither blocks nor log segments found. Creating new log.
I20260806 07:56:59.895653 32701 log.cc:826] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:59.898348 32701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: No bootstrap required, opened a new log
I20260806 07:56:59.903457 32701 raft_consensus.cc:359] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:56:59.903718 32701 raft_consensus.cc:385] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:56:59.903887 32701 raft_consensus.cc:740] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671cf1daf5974e4d8b8da0ef1b23b481, State: Initialized, Role: FOLLOWER
I20260806 07:56:59.904469 32701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:56:59.905449 336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:56:59.905794 336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:56:59.906607 32701 sys_catalog.cc:565] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:56:59.917939 347 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:56:59.918150 347 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:56:59.915247 32634 master_runner.cc:187] Error getting master registration for 127.20.202.254:36665: OK, instance_id { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" instance_seqno: 1786003018539877 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:56:59.922369 32634 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:56:59.957840 32624 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } attempt: 5
Aug 06 07:56:59 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003018, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:56:59.982867 32624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Bootstrap starting.
I20260806 07:56:59.987288 32624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Neither blocks nor log segments found. Creating new log.
W20260806 07:56:59.987551 32733 tablet.cc:2406] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:56:59.990120 32624 log.cc:826] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Log is configured to *not* fsync() on all Append() calls
I20260806 07:56:59.993388 32624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: No bootstrap required, opened a new log
I20260806 07:57:00.001901 32624 raft_consensus.cc:359] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:57:00.002381 32624 raft_consensus.cc:385] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:00.002568 32624 raft_consensus.cc:740] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d63be01ac941788f8cb40b4e97a23d, State: Initialized, Role: FOLLOWER
I20260806 07:57:00.003458 32624 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:57:00.004679 350 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.005002 350 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:00.006264 32624 sys_catalog.cc:565] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:57:00.026901 32578 tablet.cc:2406] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:57:00.033242 32556 master_runner.cc:264] No leader master found from master a5d63be01ac941788f8cb40b4e97a23d
I20260806 07:57:00.033414 32556 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260806 07:57:00.036775 361 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:57:00.036943 361 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:57:00.065799 336 raft_consensus.cc:493] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:57:00.066133 336 raft_consensus.cc:515] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:57:00.068182 336 leader_election.cc:290] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5d63be01ac941788f8cb40b4e97a23d (127.20.202.254:36665), c130e294e4ab48bdaa537108a5c9b223 (127.20.202.252:34043)
I20260806 07:57:00.069697 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d63be01ac941788f8cb40b4e97a23d" is_pre_election: true
I20260806 07:57:00.070273 32599 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671cf1daf5974e4d8b8da0ef1b23b481 in term 0.
I20260806 07:57:00.070626 32754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c130e294e4ab48bdaa537108a5c9b223" is_pre_election: true
I20260806 07:57:00.071157 32754 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671cf1daf5974e4d8b8da0ef1b23b481 in term 0.
I20260806 07:57:00.071257 32651 leader_election.cc:304] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671cf1daf5974e4d8b8da0ef1b23b481, a5d63be01ac941788f8cb40b4e97a23d; no voters:
I20260806 07:57:00.071830 336 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:57:00.072001 336 raft_consensus.cc:493] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:57:00.072108 336 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:00.080507 336 raft_consensus.cc:515] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:57:00.083902 32599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d63be01ac941788f8cb40b4e97a23d"
I20260806 07:57:00.084154 32599 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:00.088040 32599 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671cf1daf5974e4d8b8da0ef1b23b481 in term 1.
I20260806 07:57:00.088719 32651 leader_election.cc:304] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671cf1daf5974e4d8b8da0ef1b23b481, a5d63be01ac941788f8cb40b4e97a23d; no voters:
I20260806 07:57:00.093407 32754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c130e294e4ab48bdaa537108a5c9b223"
I20260806 07:57:00.093686 32754 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:00.097453 32754 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671cf1daf5974e4d8b8da0ef1b23b481 in term 1.
I20260806 07:57:00.098591 336 leader_election.cc:290] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [CANDIDATE]: Term 1 election: Requested vote from peers a5d63be01ac941788f8cb40b4e97a23d (127.20.202.254:36665), c130e294e4ab48bdaa537108a5c9b223 (127.20.202.252:34043)
I20260806 07:57:00.099098 336 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:00.099604 336 raft_consensus.cc:697] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 1 LEADER]: Becoming Leader. State: Replica: 671cf1daf5974e4d8b8da0ef1b23b481, State: Running, Role: LEADER
I20260806 07:57:00.100462 336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } }
I20260806 07:57:00.119468 363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671cf1daf5974e4d8b8da0ef1b23b481. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.120033 363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:00.126998 365 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:00.129611 365 catalog_manager.cc:1520] Initializing Kudu cluster ID...
W20260806 07:57:00.132073 32655 tablet.cc:2406] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:57:00.149736 32754 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 1 FOLLOWER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:57:00.155543 363 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:57:00.173435 32599 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:57:00.174667 363 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:57:00.196295 350 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671cf1daf5974e4d8b8da0ef1b23b481. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.196607 350 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:00.199638 366 mvcc.cc:204] Tried to move back new op lower bound from 7315468370522034176 to 7315468370361131008. Current Snapshot: MvccSnapshot[applied={T|T < 7315468370522034176}]
I20260806 07:57:00.203158 336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.203467 336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:00.205755 365 catalog_manager.cc:1383] Generated new cluster ID: e73b00e5314c46b2a02d771816d83d2a
I20260806 07:57:00.205860 365 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:00.225504 350 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.225821 350 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:00.228945 322 sys_catalog.cc:455] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671cf1daf5974e4d8b8da0ef1b23b481. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.229205 322 sys_catalog.cc:458] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:00.230607 363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.230834 363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:00.232731 367 mvcc.cc:204] Tried to move back new op lower bound from 7315468370522034176 to 7315468370361131008. Current Snapshot: MvccSnapshot[applied={T|T < 7315468370522034176}]
I20260806 07:57:00.238142 322 sys_catalog.cc:455] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } }
I20260806 07:57:00.238346 322 sys_catalog.cc:458] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:00.257081 365 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:00.259176 365 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:00.276620 365 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: Generated new TSK 0
I20260806 07:57:00.278687 365 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:00.471843 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:00.847070 379 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:00.847373 379 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:00.847523 379 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:00.856289 379 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:00.856519 379 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:00.856509 333 catalog_manager.cc:1295] Loaded cluster ID: e73b00e5314c46b2a02d771816d83d2a
I20260806 07:57:00.856622 333 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: loading cluster ID for follower catalog manager: success
I20260806 07:57:00.859315 333 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: acquiring CA information for follower catalog manager: success
I20260806 07:57:00.861171 333 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:00.867741 379 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:00.869401 379 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:00.870824 379 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:00.880862 387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:00.880973 386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:00.881237 389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:00.882064 379 server_base.cc:1061] running on GCE node
I20260806 07:57:00.882633 379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:00.883574 379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:00.884778 379 hybrid_clock.cc:648] HybridClock initialized: now 1786003020884734 us; error 42 us; skew 500 ppm
I20260806 07:57:00.885110 379 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:00 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003020, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:00.888576 379 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:00.890345 379 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:57:00.891088 379 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:00.891229 379 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:00.891566 379 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:00.894289 379 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:00.895108 379 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:00.900679 379 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:57:00.904237 396 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:00.905689 379 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:00.905849 379 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:00.906075 379 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:00.927966 379 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:00.936794 379 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:00.937146 379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:00.938378 379 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:00.940212 379 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:00.940289 379 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:00.940415 379 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:00.940521 379 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:00.978180 379 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44575
I20260806 07:57:00.978283 509 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44575 every 8 connection(s)
I20260806 07:57:00.979892 379 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:00.980337 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 379
I20260806 07:57:00.980578 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003020, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003020, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003020, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:01.016222 512 heartbeater.cc:344] Connected to a master server at 127.20.202.253:44487
I20260806 07:57:01.016250 510 heartbeater.cc:344] Connected to a master server at 127.20.202.252:34043
I20260806 07:57:01.016546 511 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36665
I20260806 07:57:01.016724 510 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:01.016718 512 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:01.016719 511 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:01.017849 512 heartbeater.cc:507] Master 127.20.202.253:44487 requested a full tablet report, sending...
I20260806 07:57:01.017886 510 heartbeater.cc:507] Master 127.20.202.252:34043 requested a full tablet report, sending...
I20260806 07:57:01.017917 511 heartbeater.cc:507] Master 127.20.202.254:36665 requested a full tablet report, sending...
I20260806 07:57:01.020505 32589 ts_manager.cc:194] Registered new tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:01.020525 32744 ts_manager.cc:194] Registered new tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:01.022109 32666 ts_manager.cc:194] Registered new tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:01.024001 32666 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:36825
I20260806 07:57:01.028879 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:57:01.039304 361 catalog_manager.cc:1295] Loaded cluster ID: e73b00e5314c46b2a02d771816d83d2a
I20260806 07:57:01.039433 361 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: loading cluster ID for follower catalog manager: success
I20260806 07:57:01.043051 361 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: acquiring CA information for follower catalog manager: success
I20260806 07:57:01.045490 361 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for kudu/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:01.170686 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:43749
--webserver_interface=127.20.202.251
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:01.523033 524 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:01.523422 524 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:01.523569 524 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:01.532522 524 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:01.532665 524 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:01.532793 524 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:01.532877 524 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:01.543929 524 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:43749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:01.545977 524 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:01.547523 524 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:01.556324 530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:01.556406 531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:01.557847 533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:01.558182 524 server_base.cc:1061] running on GCE node
I20260806 07:57:01.558945 524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:01.560396 524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:01.561717 524 hybrid_clock.cc:648] HybridClock initialized: now 1786003021561662 us; error 63 us; skew 500 ppm
I20260806 07:57:01.562070 524 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003021, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:01.565464 524 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:57:01.567252 524 webserver.cc:533] Webserver started at http://127.20.202.251:39687/ using document root <none> and password file <none>
I20260806 07:57:01.568038 524 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:01.568184 524 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:01.568557 524 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:01.571169 524 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "04084208a0894877b95f52707ff54c48"
format_stamp: "Formatted at 2026-08-06 07:57:01 on dist-test-slave-g350"
I20260806 07:57:01.572009 524 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "04084208a0894877b95f52707ff54c48"
format_stamp: "Formatted at 2026-08-06 07:57:01 on dist-test-slave-g350"
I20260806 07:57:01.577780 524 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:57:01.581691 540 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:01.583467 524 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:57:01.583662 524 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "04084208a0894877b95f52707ff54c48"
format_stamp: "Formatted at 2026-08-06 07:57:01 on dist-test-slave-g350"
I20260806 07:57:01.583928 524 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:01.607041 524 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:01.611425 524 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:01.611750 524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:01.632880 524 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:43749
I20260806 07:57:01.632918 592 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:43749 every 8 connection(s)
I20260806 07:57:01.634601 524 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:57:01.638996 593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:01.643097 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 524
I20260806 07:57:01.645996 593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003021, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:57:01.667308 593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003021, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:57:01.680982 593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003021, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:01.693964 593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003021, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:57:01.709851 593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Bootstrap starting.
I20260806 07:57:01.713189 593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:01.714268 593 log.cc:826] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:01.716878 593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: No bootstrap required, opened a new log
I20260806 07:57:01.722141 593 raft_consensus.cc:359] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } }
I20260806 07:57:01.722417 593 raft_consensus.cc:385] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:01.722496 593 raft_consensus.cc:740] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04084208a0894877b95f52707ff54c48, State: Initialized, Role: FOLLOWER
I20260806 07:57:01.723094 593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } }
I20260806 07:57:01.723940 602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } } }
I20260806 07:57:01.724272 602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:01.724900 593 sys_catalog.cc:565] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:57:01.734417 613 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:57:01.734560 613 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:57:01.742347 524 master_runner.cc:429] Detected that this master 04084208a0894877b95f52707ff54c48 is joining an existing cluster
I20260806 07:57:01.742458 524 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:57:01.743445 524 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:57:01.743834 524 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:57:01.744061 524 tablet_replica.cc:333] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: stopping tablet replica
I20260806 07:57:01.745757 524 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:43749
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003019, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:57:01.817155 32666 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:43749: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "04084208a0894877b95f52707ff54c48"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 43749
}
attrs {
promote: true
}
}
dest_uuid: "671cf1daf5974e4d8b8da0ef1b23b481"
cas_config_opid_index: -1
I20260806 07:57:01.818984 32666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } }
I20260806 07:57:01.822724 32599 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:57:01.823455 32754 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 1 FOLLOWER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:57:01.824120 32652 consensus_peers.cc:597] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 -> Peer 04084208a0894877b95f52707ff54c48 (127.20.202.251:43749): Couldn't send request to peer 04084208a0894877b95f52707ff54c48. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:57:01.824580 624 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:57:01.825028 380 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:57:01.828347 624 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 04084208a0894877b95f52707ff54c48 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.829933 32599 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 04084208a0894877b95f52707ff54c48 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.832536 624 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:43749
I20260806 07:57:01.833480 524 master.cc:562] Master@127.20.202.251:43749 shutting down...
I20260806 07:57:01.851018 628 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.852603 628 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:01.856511 380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.856882 380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:01.860474 32754 raft_consensus.cc:2955] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 04084208a0894877b95f52707ff54c48 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.862896 629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.862993 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.863232 629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:01.863318 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:01.865669 631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:01.866083 631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:01.894526 524 master.cc:584] Master@127.20.202.251:43749 shutdown complete.
I20260806 07:57:01.894621 524 master_runner.cc:306] Clearing existing system tablet
I20260806 07:57:01.895032 524 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:01.895136 524 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:01.897435 524 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:57:01.899017 633 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:01.899374 524 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:01.899631 524 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 07:57:01.899722 524 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "04084208a0894877b95f52707ff54c48"
format_stamp: "Formatted at 2026-08-06 07:57:01 on dist-test-slave-g350"
I20260806 07:57:01.903046 524 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:57:01.907967 524 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:57:01.908128 524 log.cc:1199] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:57:01.908572 524 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Deleting consensus metadata
I20260806 07:57:01.908811 524 master_runner.cc:316] Copying system tablet from 127.20.202.253:44487
I20260806 07:57:01.911682 524 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.253:44487
I20260806 07:57:01.925791 32686 tablet_copy_service.cc:140] P 671cf1daf5974e4d8b8da0ef1b23b481: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 04084208a0894877b95f52707ff54c48 ({username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:33808)
I20260806 07:57:01.926178 32686 tablet_copy_service.cc:161] P 671cf1daf5974e4d8b8da0ef1b23b481: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 04084208a0894877b95f52707ff54c48 at {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:33808: session id = 04084208a0894877b95f52707ff54c48-00000000000000000000000000000000
I20260806 07:57:01.930476 32686 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:57:01.933135 524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:57:01.939553 524 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: tablet copy: Starting download of 0 data blocks...
I20260806 07:57:01.939962 524 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: tablet copy: Starting download of 1 WAL segments...
I20260806 07:57:01.944823 524 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:57:01.948720 32686 tablet_copy_service.cc:342] P 671cf1daf5974e4d8b8da0ef1b23b481: Request end of tablet copy session 04084208a0894877b95f52707ff54c48-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:33808
I20260806 07:57:01.948956 32686 tablet_copy_service.cc:434] P 671cf1daf5974e4d8b8da0ef1b23b481: ending tablet copy session 04084208a0894877b95f52707ff54c48-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 04084208a0894877b95f52707ff54c48
W20260806 07:57:01.957747 524 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:57:01.962291 524 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:01.965453 641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:01.966441 640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:01.967244 643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:01.967876 524 server_base.cc:1061] running on GCE node
I20260806 07:57:01.968218 524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:01.968868 524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:01.970093 524 hybrid_clock.cc:648] HybridClock initialized: now 1786003021970040 us; error 39 us; skew 500 ppm
I20260806 07:57:01.970346 524 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:01 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003021, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:01.972591 524 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:57:01.973989 524 webserver.cc:533] Webserver started at http://127.20.202.251:39773/ using document root <none> and password file <none>
I20260806 07:57:01.974337 524 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:01.974448 524 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:01.976764 524 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 07:57:01.978770 650 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:01.979272 524 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 07:57:01.979413 524 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "04084208a0894877b95f52707ff54c48"
format_stamp: "Formatted at 2026-08-06 07:57:01 on dist-test-slave-g350"
I20260806 07:57:01.979594 524 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:01.994261 524 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:01.997658 524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:02.020018 524 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:43749
I20260806 07:57:02.019956 702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:43749 every 8 connection(s)
I20260806 07:57:02.021071 524 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:57:02.027240 512 heartbeater.cc:499] Master 127.20.202.253:44487 was elected leader, sending a full tablet report...
I20260806 07:57:02.027632 703 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:57:02.030633 703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Bootstrap starting.
I20260806 07:57:02.047484 703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:57:02.048039 703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: Bootstrap complete.
I20260806 07:57:02.049382 703 raft_consensus.cc:359] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } }
I20260806 07:57:02.049669 703 raft_consensus.cc:740] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 04084208a0894877b95f52707ff54c48, State: Initialized, Role: LEARNER
I20260806 07:57:02.049949 703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } }
I20260806 07:57:02.051010 707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:02.051370 703 sys_catalog.cc:565] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:02.051584 707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:57:02.062934 719 catalog_manager.cc:1295] Loaded cluster ID: e73b00e5314c46b2a02d771816d83d2a
I20260806 07:57:02.063078 719 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: loading cluster ID for follower catalog manager: success
I20260806 07:57:02.066009 719 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: acquiring CA information for follower catalog manager: success
I20260806 07:57:02.068005 719 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:02.109470 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 379
Aug 06 07:57:02 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:02.129434 505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:02.218816 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 379
I20260806 07:57:02.238490 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44575
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:02.275238 677 raft_consensus.cc:1217] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:57:02.276046 707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671cf1daf5974e4d8b8da0ef1b23b481. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:02.276317 707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: This master's current role is: LEARNER
W20260806 07:57:02.593030 722 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:02.593351 722 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:02.593444 722 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:02.602150 722 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:02.602355 722 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:02.613433 722 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:02.615111 722 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:02.616482 722 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:02.626360 728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:02.626523 729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:02.627436 731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:02.627796 722 server_base.cc:1061] running on GCE node
I20260806 07:57:02.628533 722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:02.629554 722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:02.630839 722 hybrid_clock.cc:648] HybridClock initialized: now 1786003022630769 us; error 55 us; skew 500 ppm
I20260806 07:57:02.631110 722 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:02 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:02.634509 722 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:02.636113 722 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:57:02.636798 722 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:02.636951 722 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:02.642810 722 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:57:02.645776 738 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:02.647279 722 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:57:02.647447 722 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:02.648219 722 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:02.664168 722 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:02.667975 722 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:02.668257 722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:02.669282 722 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:02.670949 722 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:02.671038 722 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:02.671167 722 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:02.671267 722 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:02.707209 722 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44575
I20260806 07:57:02.707270 853 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44575 every 8 connection(s)
I20260806 07:57:02.708969 722 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:02.712370 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 722
Aug 06 07:57:02 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:57:02 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:57:02 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:02 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:02.761448 380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: true } } }
I20260806 07:57:02.761821 380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:02.763005 856 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36665
I20260806 07:57:02.763464 856 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:02.764680 856 heartbeater.cc:507] Master 127.20.202.254:36665 requested a full tablet report, sending...
I20260806 07:57:02.767180 32589 ts_manager.cc:194] Re-registered known tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:02.769496 857 heartbeater.cc:344] Connected to a master server at 127.20.202.253:44487
I20260806 07:57:02.769690 857 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:02.770238 857 heartbeater.cc:507] Master 127.20.202.253:44487 requested a full tablet report, sending...
I20260806 07:57:02.770692 855 heartbeater.cc:344] Connected to a master server at 127.20.202.252:34043
I20260806 07:57:02.770848 855 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:02.771389 855 heartbeater.cc:507] Master 127.20.202.252:34043 requested a full tablet report, sending...
I20260806 07:57:02.772382 854 heartbeater.cc:344] Connected to a master server at 127.20.202.251:43749
I20260806 07:57:02.772516 854 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:02.772846 32744 ts_manager.cc:194] Re-registered known tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:02.772953 854 heartbeater.cc:507] Master 127.20.202.251:43749 requested a full tablet report, sending...
I20260806 07:57:02.774010 32666 ts_manager.cc:194] Re-registered known tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:02.774262 866 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481: attempting to promote NON_VOTER 04084208a0894877b95f52707ff54c48 to VOTER
I20260806 07:57:02.774996 667 ts_manager.cc:194] Registered new tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:02.775303 32666 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:41993
I20260806 07:57:02.775390 866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } }
I20260806 07:57:02.778297 677 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 1 LEARNER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:57:02.778543 32754 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 1 FOLLOWER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:57:02.779040 380 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:02.779639 380 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:02.780201 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:57:02.782037 32599 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Refusing update from remote peer 671cf1daf5974e4d8b8da0ef1b23b481: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:57:02.782967 380 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:02.784468 32754 raft_consensus.cc:2955] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 04084208a0894877b95f52707ff54c48 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.785363 32599 raft_consensus.cc:2955] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 04084208a0894877b95f52707ff54c48 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.788857 871 sys_catalog.cc:455] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.789203 871 sys_catalog.cc:458] T 00000000000000000000000000000000 P c130e294e4ab48bdaa537108a5c9b223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:02.783656 866 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 04084208a0894877b95f52707ff54c48 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.790657 872 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.790917 872 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:02.793560 866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.793687 870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.794029 870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:02.794798 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:57:02.794968 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32556
I20260806 07:57:02.794294 677 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 04084208a0894877b95f52707ff54c48 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.795974 32619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:02.797820 867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.798219 867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:02.798617 866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:02.800782 876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.801397 876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04084208a0894877b95f52707ff54c48 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:02.801748 870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:02.802003 870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:02.912487 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32556
I20260806 07:57:02.935984 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36665
--webserver_interface=127.20.202.254
--webserver_port=32961
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:03.296411 877 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:03.296851 877 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:03.297031 877 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:03.305930 877 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:03.306025 877 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:03.306084 877 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:03.306133 877 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:57:03.311002 32651 consensus_peers.cc:597] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 -> Peer a5d63be01ac941788f8cb40b4e97a23d (127.20.202.254:36665): Couldn't send request to peer a5d63be01ac941788f8cb40b4e97a23d. Status: Network error: Client connection negotiation failed: client connection to 127.20.202.254:36665: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 07:57:03.318171 877 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=32961
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:03.319751 877 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:03.321201 877 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:03.329648 885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:03.329742 884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:03.330626 887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:03.331409 877 server_base.cc:1061] running on GCE node
I20260806 07:57:03.332242 877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:03.333770 877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:03.334991 877 hybrid_clock.cc:648] HybridClock initialized: now 1786003023334939 us; error 54 us; skew 500 ppm
I20260806 07:57:03.335352 877 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:03 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003023, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:03.338845 877 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:03.340565 877 webserver.cc:533] Webserver started at http://127.20.202.254:32961/ using document root <none> and password file <none>
I20260806 07:57:03.341338 877 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:03.341480 877 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:03.347340 877 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:57:03.350452 895 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:03.352094 877 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:03.352290 877 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "a5d63be01ac941788f8cb40b4e97a23d"
format_stamp: "Formatted at 2026-08-06 07:56:58 on dist-test-slave-g350"
I20260806 07:57:03.353130 877 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:03.386335 877 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:03.390506 877 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:03.390790 877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:03.413297 877 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:36665
I20260806 07:57:03.413349 946 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:36665 every 8 connection(s)
I20260806 07:57:03.415083 877 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:03.419683 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 877
I20260806 07:57:03.419981 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 722
I20260806 07:57:03.422806 947 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:57:03.426412 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Bootstrap starting.
I20260806 07:57:03.436553 849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:03.448256 947 log.cc:826] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:03.462850 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:57:03.463595 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: Bootstrap complete.
I20260806 07:57:03.474629 947 raft_consensus.cc:359] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } }
I20260806 07:57:03.475713 947 raft_consensus.cc:740] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d63be01ac941788f8cb40b4e97a23d, State: Initialized, Role: FOLLOWER
I20260806 07:57:03.476720 947 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } }
I20260806 07:57:03.477758 953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:03.478124 953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:03.479275 947 sys_catalog.cc:565] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:03.491820 964 catalog_manager.cc:1295] Loaded cluster ID: e73b00e5314c46b2a02d771816d83d2a
I20260806 07:57:03.492034 964 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: loading cluster ID for follower catalog manager: success
I20260806 07:57:03.496920 964 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: acquiring CA information for follower catalog manager: success
I20260806 07:57:03.500293 964 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:03.559281 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 722
I20260806 07:57:03.586555 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44575
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:03.746613 873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:03.746928 873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:03.759032 953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671cf1daf5974e4d8b8da0ef1b23b481. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:03.759289 953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d63be01ac941788f8cb40b4e97a23d [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:57:03.962570 971 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:03.962920 971 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:03.963079 971 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:03.971359 971 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:03.971621 971 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:03.982664 971 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:03.984475 971 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:03.986083 971 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:03.996011 980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:03.996407 979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:03.996428 982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:03.996999 971 server_base.cc:1061] running on GCE node
I20260806 07:57:03.997581 971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:03.998595 971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:03.999960 971 hybrid_clock.cc:648] HybridClock initialized: now 1786003023999884 us; error 62 us; skew 500 ppm
I20260806 07:57:04.000317 971 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003024, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:04.003732 971 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:04.005499 971 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:57:04.006278 971 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:04.006420 971 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:04.012323 971 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:57:04.015182 989 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:04.016621 971 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:57:04.016811 971 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:04.017539 971 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:04.060743 971 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:04.065003 971 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:04.065323 971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:04.066463 971 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:04.068145 971 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:04.068236 971 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:04.068387 971 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:04.068462 971 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:04.104728 971 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44575
I20260806 07:57:04.104779 1103 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44575 every 8 connection(s)
I20260806 07:57:04.106379 971 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:04.110116 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 971
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786002999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003024, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003024, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003024, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003024, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:04.156226 1107 heartbeater.cc:344] Connected to a master server at 127.20.202.253:44487
I20260806 07:57:04.156610 1107 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:04.157893 1107 heartbeater.cc:507] Master 127.20.202.253:44487 requested a full tablet report, sending...
I20260806 07:57:04.160097 32666 ts_manager.cc:194] Re-registered known tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:04.160990 1106 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36665
I20260806 07:57:04.161126 1106 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:04.161295 32666 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:52523
I20260806 07:57:04.161571 1106 heartbeater.cc:507] Master 127.20.202.254:36665 requested a full tablet report, sending...
I20260806 07:57:04.163942 912 ts_manager.cc:194] Registered new tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:04.169833 1104 heartbeater.cc:344] Connected to a master server at 127.20.202.251:43749
I20260806 07:57:04.169983 1104 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:04.170383 1104 heartbeater.cc:507] Master 127.20.202.251:43749 requested a full tablet report, sending...
I20260806 07:57:04.171625 1105 heartbeater.cc:344] Connected to a master server at 127.20.202.252:34043
I20260806 07:57:04.171764 1105 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:04.171849 667 ts_manager.cc:194] Re-registered known tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:04.172050 1105 heartbeater.cc:507] Master 127.20.202.252:34043 requested a full tablet report, sending...
I20260806 07:57:04.173111 32744 ts_manager.cc:194] Re-registered known tserver with Master: 6184913e1e14415d88b200f5a75cf01b (127.20.202.193:44575)
I20260806 07:57:04.271149 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 971
W20260806 07:57:04.272583 1121 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:04.268869 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.20.202.193:44575)
0806 07:57:04.269640 (+ 771us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:04.269651 (+ 11us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:04.269690 (+ 39us) server_negotiation.cc:408] Connection header received
0806 07:57:04.269757 (+ 67us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:04.269764 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:04.269985 (+ 221us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:04.270171 (+ 186us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:04.271152 (+ 981us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60448: BlockingRecv error: recv got EOF from 127.0.0.1:60448 (error 108)
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":417,"threads_started":1}
W20260806 07:57:04.273690 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 971: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:44575: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:57:04.273849 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 971
I20260806 07:57:04.293035 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44575
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:04.328379 972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a5d63be01ac941788f8cb40b4e97a23d" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36665 } } peers { permanent_uuid: "671cf1daf5974e4d8b8da0ef1b23b481" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 44487 } } peers { permanent_uuid: "c130e294e4ab48bdaa537108a5c9b223" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 34043 } } peers { permanent_uuid: "04084208a0894877b95f52707ff54c48" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 43749 } attrs { promote: false } } }
I20260806 07:57:04.328655 972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671cf1daf5974e4d8b8da0ef1b23b481 [sys.catalog]: This master's current role is: LEADER
W20260806 07:57:04.643409 1123 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:04.643791 1123 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:04.643956 1123 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:57:04.644048 1123 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:04.652760 1123 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:04.653019 1123 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:04.664322 1123 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:04.666056 1123 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:04.667464 1123 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:04.677335 1131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:04.677384 1132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:04.677932 1134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:04.679562 1123 server_base.cc:1061] running on GCE node
I20260806 07:57:04.680120 1123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:04.681206 1123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:04.682456 1123 hybrid_clock.cc:648] HybridClock initialized: now 1786003024682378 us; error 54 us; skew 500 ppm
I20260806 07:57:04.682857 1123 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:04 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003024, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:04.686266 1123 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:04.687925 1123 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:57:04.688696 1123 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:04.688819 1123 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:04.694624 1123 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:57:04.697422 1141 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:04.698904 1123 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:57:04.699115 1123 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:04.699828 1123 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:04.727828 1123 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:04.731587 1123 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:04.731891 1123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:04.732893 1123 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:04.734512 1123 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:04.734632 1123 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:04.734797 1123 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:04.734858 1123 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:57:04.735203 1123 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:57:04.735263 1123 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:57:04.735356 1123 fault_injection.cc:59] injecting fault for kudu (pid 1123): FLAGS_fault_before_start (process will exit)
I20260806 07:57:04.756152 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:44575
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043
--webserver_port=37199
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:05.123134 1149 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:05.123498 1149 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:05.123653 1149 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:05.132304 1149 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:05.132571 1149 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:05.143755 1149 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=37199
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:36665,127.20.202.253:44487,127.20.202.252:34043,127.20.202.251:43749
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:05.145500 1149 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:05.147222 1149 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:05.158391 1157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:05.158448 1158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:05.159505 1160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:05.160666 1149 server_base.cc:1061] running on GCE node
I20260806 07:57:05.161307 1149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:05.162910 1149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:05.164202 1149 hybrid_clock.cc:648] HybridClock initialized: now 1786003025164139 us; error 56 us; skew 500 ppm
I20260806 07:57:05.164489 1149 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:05 dist-test-slave-g350 krb5kdc[32251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003025, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:05.167906 1149 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:05.169571 1149 webserver.cc:533] Webserver started at http://127.20.202.193:37199/ using document root <none> and password file <none>
I20260806 07:57:05.170361 1149 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:05.170456 1149 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:05.176007 1149 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 07:57:05.178949 1167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:05.180596 1149 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:57:05.180809 1149 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "6184913e1e14415d88b200f5a75cf01b"
format_stamp: "Formatted at 2026-08-06 07:57:00 on dist-test-slave-g350"
I20260806 07:57:05.181574 1149 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:05.204854 1149 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:05.208969 1149 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:05.209211 1149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:05.210325 1149 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:05.212018 1149 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:05.212144 1149 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:05.212280 1149 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:05.212373 1149 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:05.249490 1149 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:44575
I20260806 07:57:05.249523 1280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:44575 every 8 connection(s)
I20260806 07:57:05.251120 1149 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:05.251538 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1149
I20260806 07:57:05.251835 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1149
W20260806 07:57:05.270850 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1149: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:44575: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:57:05.271019 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1149
I20260806 07:57:05.290359 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 877
W20260806 07:57:05.290977 1151 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:05.267883 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:50361 (local address 127.20.202.254:36665)
0806 07:57:05.268078 (+ 195us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:05.268085 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:05.289971 (+ 21886us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:50361: BlockingRecv error: recv got EOF from 127.20.202.193:50361 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 07:57:05.291056 1286 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:05.255283 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:49409 (local address 127.20.202.252:34043)
0806 07:57:05.265405 (+ 10122us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:05.265414 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:05.266936 (+ 1522us) server_negotiation.cc:408] Connection header received
0806 07:57:05.268860 (+ 1924us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:05.268866 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:05.268964 (+ 98us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:05.269074 (+ 110us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:05.290179 (+ 21105us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:49409: BlockingRecv error: recv got EOF from 127.20.202.193:49409 (error 108)
Metrics: {"server-negotiator.queue_time_us":9953,"thread_start_us":9842,"threads_started":1}
W20260806 07:57:05.291194 1287 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:05.256353 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:41295 (local address 127.20.202.251:43749)
0806 07:57:05.278190 (+ 21837us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:05.278199 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:05.278230 (+ 31us) server_negotiation.cc:408] Connection header received
0806 07:57:05.278310 (+ 80us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:05.278319 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:05.278439 (+ 120us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:05.278590 (+ 151us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:05.290476 (+ 11886us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:41295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":21660,"thread_start_us":21522,"threads_started":1}
W20260806 07:57:05.292174 1294 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:05.269120 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:41093 (local address 127.20.202.253:44487)
0806 07:57:05.278459 (+ 9339us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:05.278464 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:05.290011 (+ 11547us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:41093: BlockingRecv error: recv got EOF from 127.20.202.193:41093 (error 108)
Metrics: {"server-negotiator.queue_time_us":9223,"thread_start_us":9116,"threads_started":1}
W20260806 07:57:05.293181 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 877: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:36665: server requires authentication, but client does not have Kerberos credentials available
W20260806 07:57:05.293243 1151 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:05.291744 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.20.202.254:36665)
0806 07:57:05.291948 (+ 204us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:05.291954 (+ 6us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:05.291978 (+ 24us) server_negotiation.cc:408] Connection header received
0806 07:57:05.292037 (+ 59us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:05.292043 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:05.292107 (+ 64us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:05.292220 (+ 113us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:05.293056 (+ 836us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60044: BlockingRecv error: recv got EOF from 127.0.0.1:60044 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 07:57:05.293321 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 877
I20260806 07:57:05.315546 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32634
I20260806 07:57:05.316850 32696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:05.433868 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32634
I20260806 07:57:05.457886 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 32712
I20260806 07:57:05.459245 307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:05.571460 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 32712
I20260806 07:57:05.593784 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 524
I20260806 07:57:05.595228 698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:05.709147 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 524
Shutting down hive metastore.
2026-08-06T07:57:06Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/10 (28616 ms)
[ RUN ] ExternalMiniClusterTest.TestAddMaster/11
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:06 dist-test-slave-g350 krb5kdc[1305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:06 dist-test-slave-g350 krb5kdc[1305](info): set up 2 sockets
Aug 06 07:57:06 dist-test-slave-g350 krb5kdc[1305](info): commencing operation
krb5kdc: starting...
W20260806 07:57:08.228833 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.080s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:08 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:08Z Disabled control of system clock
2026-08-06T07:57:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:08Z Disabled control of system clock
2026-08-06T07:57:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:08Z Disabled control of system clock
2026-08-06T07:57:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:08Z Disabled control of system clock
2026-08-06T07:57:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:08Z Disabled control of system clock
WARNING: no policy specified for hive/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "hive/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Entry for principal hive/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/hive_127.0.0.1.keytab.
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf
2026-08-06 07:57:16: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/security.properties -Djava.security.krb5.conf=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskUU61cc/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
Aug 06 07:57:18 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786003038, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:57:24 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786003044, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 07:57:24 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786003044, etypes {rep=17 tkt=17 ses=17}, hive/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:25.189227 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35569
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:25.543709 1613 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:25.544114 1613 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:25.544281 1613 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:25.554075 1613 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:25.554227 1613 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:25.554361 1613 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:25.554450 1613 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:25.566390 1613 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:25.568110 1613 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:25.569533 1613 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:25.578258 1618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:25.578151 1619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:25.579561 1621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:25.579890 1613 server_base.cc:1061] running on GCE node
I20260806 07:57:25.580631 1613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:25.582347 1613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:25.583653 1613 hybrid_clock.cc:648] HybridClock initialized: now 1786003045583508 us; error 110 us; skew 500 ppm
I20260806 07:57:25.584074 1613 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:25 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003045, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:25.587802 1613 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:25.589861 1613 webserver.cc:533] Webserver started at http://127.20.202.254:44175/ using document root <none> and password file <none>
I20260806 07:57:25.590644 1613 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:25.590787 1613 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:25.591156 1613 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:25.593880 1613 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "66a12f0d487a4fa781a4cd6253ded947"
format_stamp: "Formatted at 2026-08-06 07:57:25 on dist-test-slave-g350"
I20260806 07:57:25.594748 1613 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "66a12f0d487a4fa781a4cd6253ded947"
format_stamp: "Formatted at 2026-08-06 07:57:25 on dist-test-slave-g350"
I20260806 07:57:25.600577 1613 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 07:57:25.604182 1628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:25.605564 1613 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:57:25.605814 1613 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "66a12f0d487a4fa781a4cd6253ded947"
format_stamp: "Formatted at 2026-08-06 07:57:25 on dist-test-slave-g350"
I20260806 07:57:25.606019 1613 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:25.624500 1613 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:25.632787 1613 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:25.633109 1613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:25.654345 1613 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:35569
I20260806 07:57:25.654386 1681 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:35569 every 8 connection(s)
I20260806 07:57:25.655990 1613 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:25.660465 1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:25.660503 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1613
I20260806 07:57:25.660720 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:25.666441 1682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:25 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003045, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:25.689388 1682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:57:25.690913 1630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38203: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.202.253@KRBTEST.COM; defaulting to no policy
W20260806 07:57:25.692926 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38203: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38203: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:25.742313 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } attempt: 1
W20260806 07:57:25.744441 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38203: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38203: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:25.797017 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38203
--webserver_interface=127.20.202.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:25.812709 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } attempt: 2
W20260806 07:57:25.814886 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38203: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38203: connect: Connection refused (error 111)
I20260806 07:57:25.906195 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } attempt: 3
W20260806 07:57:25.908504 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38203: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38203: connect: Connection refused (error 111)
I20260806 07:57:26.051816 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } attempt: 4
W20260806 07:57:26.054126 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:38203: Network error: Client connection negotiation failed: client connection to 127.20.202.253:38203: connect: Connection refused (error 111)
W20260806 07:57:26.170534 1691 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:26.170851 1691 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:26.170941 1691 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:26.179443 1691 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:26.179548 1691 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:26.179605 1691 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:26.179654 1691 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:26.190786 1691 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:38203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:26.192466 1691 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:26.193868 1691 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:26.202235 1696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:26.202275 1697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:26.203277 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:26.203778 1691 server_base.cc:1061] running on GCE node
I20260806 07:57:26.204546 1691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:26.206265 1691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:26.207479 1691 hybrid_clock.cc:648] HybridClock initialized: now 1786003046207332 us; error 100 us; skew 500 ppm
I20260806 07:57:26.207823 1691 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:26.210944 1691 init.cc:377] Logged in from keytab as kudu/127.20.202.253@KRBTEST.COM (short username kudu)
I20260806 07:57:26.212671 1691 webserver.cc:533] Webserver started at http://127.20.202.253:44287/ using document root <none> and password file <none>
I20260806 07:57:26.213371 1691 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:26.213464 1691 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:26.213845 1691 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:26.216339 1691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "6f6862cc5b8541188d81f82d5bd845da"
format_stamp: "Formatted at 2026-08-06 07:57:26 on dist-test-slave-g350"
I20260806 07:57:26.217118 1691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "6f6862cc5b8541188d81f82d5bd845da"
format_stamp: "Formatted at 2026-08-06 07:57:26 on dist-test-slave-g350"
I20260806 07:57:26.222786 1691 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 07:57:26.226188 1706 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:26.227645 1691 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:57:26.227790 1691 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "6f6862cc5b8541188d81f82d5bd845da"
format_stamp: "Formatted at 2026-08-06 07:57:26 on dist-test-slave-g350"
I20260806 07:57:26.227959 1691 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:26.247393 1691 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:26.251601 1691 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:26.251888 1691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:26.275183 1691 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:38203
I20260806 07:57:26.275221 1758 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:38203 every 8 connection(s)
I20260806 07:57:26.276908 1691 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:57:26.279068 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1691
I20260806 07:57:26.279289 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:57:26.281383 1759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:26.287529 1759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:26.310866 1759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.252@KRBTEST.COM" created.
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:57:26.331804 1759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:57:26.333040 1707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
W20260806 07:57:26.334910 1759 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
I20260806 07:57:26.353367 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003045, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:57:26.368161 1682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:57:26.370129 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
I20260806 07:57:26.384228 1759 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 1
W20260806 07:57:26.386400 1759 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
WARNING: no policy specified for HTTP/127.20.202.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.252@KRBTEST.COM" created.
I20260806 07:57:26.421365 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 1
W20260806 07:57:26.423604 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:26.425444 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:44527
--webserver_interface=127.20.202.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:26.454644 1759 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 2
W20260806 07:57:26.456786 1759 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
I20260806 07:57:26.491859 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 2
W20260806 07:57:26.494155 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
I20260806 07:57:26.548053 1759 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 3
W20260806 07:57:26.550320 1759 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
I20260806 07:57:26.600441 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 3
W20260806 07:57:26.603086 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
I20260806 07:57:26.693572 1759 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 4
W20260806 07:57:26.695717 1759 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
W20260806 07:57:26.779673 1769 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:26.780058 1769 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:26.780207 1769 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260806 07:57:26.780349 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 4
W20260806 07:57:26.782522 1682 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.252:44527: Network error: Client connection negotiation failed: client connection to 127.20.202.252:44527: connect: Connection refused (error 111)
W20260806 07:57:26.789196 1769 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:26.789335 1769 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:26.789438 1769 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:26.789516 1769 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:26.800877 1769 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.252:44527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.202.252
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:26.802685 1769 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:26.804134 1769 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:26.813818 1775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:26.813915 1774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:26.814688 1778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:26.815887 1769 server_base.cc:1061] running on GCE node
I20260806 07:57:26.816773 1769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:26.818840 1769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:26.820091 1769 hybrid_clock.cc:648] HybridClock initialized: now 1786003046819950 us; error 103 us; skew 500 ppm
I20260806 07:57:26.820513 1769 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:26.824174 1769 init.cc:377] Logged in from keytab as kudu/127.20.202.252@KRBTEST.COM (short username kudu)
I20260806 07:57:26.826074 1769 webserver.cc:533] Webserver started at http://127.20.202.252:35065/ using document root <none> and password file <none>
I20260806 07:57:26.826849 1769 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:26.826982 1769 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:26.827349 1769 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:26.830024 1769 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data/instance:
uuid: "b9a97a8d315d4084b8fe58d9a0d0971b"
format_stamp: "Formatted at 2026-08-06 07:57:26 on dist-test-slave-g350"
I20260806 07:57:26.830822 1769 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal/instance:
uuid: "b9a97a8d315d4084b8fe58d9a0d0971b"
format_stamp: "Formatted at 2026-08-06 07:57:26 on dist-test-slave-g350"
I20260806 07:57:26.836440 1769 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 07:57:26.839898 1785 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:26.841334 1769 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 07:57:26.841511 1769 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal
uuid: "b9a97a8d315d4084b8fe58d9a0d0971b"
format_stamp: "Formatted at 2026-08-06 07:57:26 on dist-test-slave-g350"
I20260806 07:57:26.841774 1769 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:26.888482 1769 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:26.892663 1769 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:26.893005 1769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:26.915516 1769 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.252:44527
I20260806 07:57:26.915536 1836 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.252:44527 every 8 connection(s)
I20260806 07:57:26.917342 1769 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/data/info.pb
I20260806 07:57:26.917696 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1769
I20260806 07:57:26.917904 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-2/wal/instance
I20260806 07:57:26.923375 1837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:26.936139 1837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:57:26.959779 1837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:26.973810 1837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.252@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:26.991328 1837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: Bootstrap starting.
I20260806 07:57:26.994572 1837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:26.994923 1759 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 5
I20260806 07:57:26.995940 1837 log.cc:826] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:26.999127 1837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: No bootstrap required, opened a new log
Aug 06 07:57:26 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003046, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.253@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:27.005235 1837 raft_consensus.cc:359] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.005548 1837 raft_consensus.cc:385] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:27.005739 1837 raft_consensus.cc:740] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a97a8d315d4084b8fe58d9a0d0971b, State: Initialized, Role: FOLLOWER
I20260806 07:57:27.006455 1837 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.008083 1846 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.008385 1846 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.009158 1837 sys_catalog.cc:565] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:27.013959 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: Bootstrap starting.
I20260806 07:57:27.018965 1759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:27.020184 1759 log.cc:826] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:27.023290 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: No bootstrap required, opened a new log
W20260806 07:57:27.024507 1857 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:57:27.024670 1857 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:57:27.022051 1769 master_runner.cc:187] Error getting master registration for 127.20.202.254:35569: OK, instance_id { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" instance_seqno: 1786003045648432 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:57:27.028813 1769 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:57:27.029949 1759 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.030233 1759 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:27.030359 1759 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6862cc5b8541188d81f82d5bd845da, State: Initialized, Role: FOLLOWER
I20260806 07:57:27.030912 1759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.031889 1860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.032202 1860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.032915 1759 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:57:27.052721 1871 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:57:27.053016 1871 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:57:27.046922 1691 master_runner.cc:187] Error getting master registration for 127.20.202.254:35569: OK, instance_id { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" instance_seqno: 1786003045648432 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:57:27.058542 1691 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:57:27.059756 1682 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } attempt: 5
Aug 06 07:57:27 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003045, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:27.084316 1682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Bootstrap starting.
I20260806 07:57:27.089455 1682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:27.091009 1682 log.cc:826] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:27.094408 1682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: No bootstrap required, opened a new log
I20260806 07:57:27.103583 1682 raft_consensus.cc:359] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.104081 1682 raft_consensus.cc:385] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:27.104264 1682 raft_consensus.cc:740] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a12f0d487a4fa781a4cd6253ded947, State: Initialized, Role: FOLLOWER
I20260806 07:57:27.105260 1682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.106451 1875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.106869 1875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.107941 1682 sys_catalog.cc:565] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:27.108150 1875 raft_consensus.cc:493] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:57:27.108425 1875 raft_consensus.cc:515] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.111254 1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a12f0d487a4fa781a4cd6253ded947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f6862cc5b8541188d81f82d5bd845da" is_pre_election: true
I20260806 07:57:27.111891 1734 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a12f0d487a4fa781a4cd6253ded947 in term 0.
I20260806 07:57:27.112864 1630 leader_election.cc:304] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a12f0d487a4fa781a4cd6253ded947, 6f6862cc5b8541188d81f82d5bd845da; no voters:
I20260806 07:57:27.123826 1875 leader_election.cc:290] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f6862cc5b8541188d81f82d5bd845da (127.20.202.253:38203), b9a97a8d315d4084b8fe58d9a0d0971b (127.20.202.252:44527)
I20260806 07:57:27.124192 1812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a12f0d487a4fa781a4cd6253ded947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" is_pre_election: true
I20260806 07:57:27.124402 1875 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:57:27.124581 1875 raft_consensus.cc:493] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:57:27.124679 1875 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:27.124783 1812 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a12f0d487a4fa781a4cd6253ded947 in term 0.
I20260806 07:57:27.130848 1875 raft_consensus.cc:515] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.134855 1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a12f0d487a4fa781a4cd6253ded947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f6862cc5b8541188d81f82d5bd845da"
I20260806 07:57:27.135108 1734 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:57:27.136384 1634 tablet.cc:2406] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:57:27.139000 1734 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a12f0d487a4fa781a4cd6253ded947 in term 1.
I20260806 07:57:27.143709 1630 leader_election.cc:304] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a12f0d487a4fa781a4cd6253ded947, 6f6862cc5b8541188d81f82d5bd845da; no voters:
I20260806 07:57:27.144300 1877 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 07:57:27.145305 1791 tablet.cc:2406] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:57:27.145738 1812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a12f0d487a4fa781a4cd6253ded947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b"
I20260806 07:57:27.145998 1812 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:27.149910 1812 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a12f0d487a4fa781a4cd6253ded947 in term 1.
I20260806 07:57:27.150647 1875 leader_election.cc:290] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [CANDIDATE]: Term 1 election: Requested vote from peers 6f6862cc5b8541188d81f82d5bd845da (127.20.202.253:38203), b9a97a8d315d4084b8fe58d9a0d0971b (127.20.202.252:44527)
I20260806 07:57:27.154731 1877 raft_consensus.cc:697] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 LEADER]: Becoming Leader. State: Replica: 66a12f0d487a4fa781a4cd6253ded947, State: Running, Role: LEADER
I20260806 07:57:27.155773 1877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } }
I20260806 07:57:27.172551 1875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a12f0d487a4fa781a4cd6253ded947. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.172849 1875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:27.183478 1888 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:27.187613 1890 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 07:57:27.203191 1888 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:27.218394 1812 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:57:27.218586 1734 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:57:27.219595 1875 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:57:27.220755 1875 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 07:57:27.255050 1712 tablet.cc:2406] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:57:27.267225 1846 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a12f0d487a4fa781a4cd6253ded947. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.267506 1846 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.270147 1892 mvcc.cc:204] Tried to move back new op lower bound from 7315468481395335168 to 7315468481156251648. Current Snapshot: MvccSnapshot[applied={T|T < 7315468481395335168}]
I20260806 07:57:27.274168 1882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.274520 1882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:27.276731 1894 mvcc.cc:204] Tried to move back new op lower bound from 7315468481395335168 to 7315468481156251648. Current Snapshot: MvccSnapshot[applied={T|T < 7315468481395335168}]
I20260806 07:57:27.282971 1888 catalog_manager.cc:1383] Generated new cluster ID: aadeb987aefa4f2b8774300d97344b0e
I20260806 07:57:27.283129 1888 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:27.283607 1846 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.283841 1846 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.297048 1860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a12f0d487a4fa781a4cd6253ded947. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.297304 1860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.299352 1882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.299576 1882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:27.305847 1893 mvcc.cc:204] Tried to move back new op lower bound from 7315468481395335168 to 7315468481156251648. Current Snapshot: MvccSnapshot[applied={T|T < 7315468481395335168}]
I20260806 07:57:27.310787 1860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } }
I20260806 07:57:27.311012 1860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:27.339859 1888 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:27.341873 1888 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:27.366350 1888 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Generated new TSK 0
I20260806 07:57:27.367630 1888 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:27.557514 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:27.970830 1904 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:27.971148 1904 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:27.971244 1904 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:27.980341 1904 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:27.980521 1904 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:27.990974 1904 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:27.992517 1904 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:27.993918 1904 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:28.003446 1911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:28.003439 1910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:28.004212 1913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:28.005414 1904 server_base.cc:1061] running on GCE node
I20260806 07:57:28.006057 1904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:28.007208 1904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:28.008446 1904 hybrid_clock.cc:648] HybridClock initialized: now 1786003048008289 us; error 120 us; skew 500 ppm
I20260806 07:57:28.008781 1904 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:28.012367 1904 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:28.014242 1904 webserver.cc:533] Webserver started at http://127.20.202.193:43637/ using document root <none> and password file <none>
I20260806 07:57:28.014978 1904 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:28.015127 1904 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:28.015529 1904 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:28.018123 1904 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:28.018930 1904 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:28.024580 1904 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 07:57:28.026564 1857 catalog_manager.cc:1295] Loaded cluster ID: aadeb987aefa4f2b8774300d97344b0e
I20260806 07:57:28.026652 1857 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: loading cluster ID for follower catalog manager: success
I20260806 07:57:28.028398 1920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:28.029981 1857 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: acquiring CA information for follower catalog manager: success
I20260806 07:57:28.030159 1904 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 07:57:28.030324 1904 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:28.030519 1904 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:28.031890 1857 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:28.049113 1904 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:28.053277 1904 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:28.053685 1904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:28.054976 1904 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:28.055590 1871 catalog_manager.cc:1295] Loaded cluster ID: aadeb987aefa4f2b8774300d97344b0e
I20260806 07:57:28.055752 1871 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: loading cluster ID for follower catalog manager: success
I20260806 07:57:28.057462 1904 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:28.057561 1904 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:28.057716 1904 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:28.057794 1904 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:28.060060 1871 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: acquiring CA information for follower catalog manager: success
I20260806 07:57:28.062961 1871 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:28.096925 1904 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37707
I20260806 07:57:28.096966 2034 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37707 every 8 connection(s)
I20260806 07:57:28.098556 1904 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:28.100914 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 1904
I20260806 07:57:28.101219 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:28.137145 2035 heartbeater.cc:344] Connected to a master server at 127.20.202.252:44527
I20260806 07:57:28.137588 2035 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:28.137715 2036 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35569
I20260806 07:57:28.137847 2036 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:28.138839 2036 heartbeater.cc:507] Master 127.20.202.254:35569 requested a full tablet report, sending...
I20260806 07:57:28.138866 2035 heartbeater.cc:507] Master 127.20.202.252:44527 requested a full tablet report, sending...
I20260806 07:57:28.139200 2037 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38203
I20260806 07:57:28.139361 2037 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:28.139838 2037 heartbeater.cc:507] Master 127.20.202.253:38203 requested a full tablet report, sending...
I20260806 07:57:28.141806 1723 ts_manager.cc:194] Registered new tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:28.141847 1801 ts_manager.cc:194] Registered new tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:28.143572 1646 ts_manager.cc:194] Registered new tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:28.145103 1646 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:55801
I20260806 07:57:28.150717 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
WARNING: no policy specified for kudu/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.251@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.251@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.251 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:28.286098 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:36615
--webserver_interface=127.20.202.251
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:28.642736 2049 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:28.643077 2049 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:28.643172 2049 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:28.652015 2049 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:28.652127 2049 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:28.652184 2049 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:28.652225 2049 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:28.663544 2049 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.251
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.251:36615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
--webserver_interface=127.20.202.251
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:28.665198 2049 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:28.666704 2049 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:28.675253 2054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:28.675314 2055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:28.676033 2057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:28.676714 2049 server_base.cc:1061] running on GCE node
I20260806 07:57:28.677502 2049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:28.679203 2049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:28.680533 2049 hybrid_clock.cc:648] HybridClock initialized: now 1786003048680364 us; error 128 us; skew 500 ppm
I20260806 07:57:28.680948 2049 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:28.684309 2049 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:57:28.686189 2049 webserver.cc:533] Webserver started at http://127.20.202.251:35315/ using document root <none> and password file <none>
I20260806 07:57:28.686971 2049 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:28.687112 2049 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:28.687476 2049 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:28.690088 2049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/instance:
uuid: "f5a5a432a06e4fc0a70244371d526dc4"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:28.690907 2049 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal/instance:
uuid: "f5a5a432a06e4fc0a70244371d526dc4"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:28.696623 2049 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:57:28.700363 2064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:28.701987 2049 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:28.702190 2049 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "f5a5a432a06e4fc0a70244371d526dc4"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:28.702415 2049 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:28.723992 2049 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:28.728438 2049 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:28.728788 2049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:28.752107 2049 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:36615
I20260806 07:57:28.752171 2116 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:36615 every 8 connection(s)
I20260806 07:57:28.754032 2049 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:57:28.758903 2117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:28.758962 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2049
I20260806 07:57:28.767381 2117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:28.787166 2117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
I20260806 07:57:28.800036 2117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
I20260806 07:57:28.813222 2117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:57:28.831089 2117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Bootstrap starting.
I20260806 07:57:28.834771 2117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:28.836107 2117 log.cc:826] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:28.839624 2117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: No bootstrap required, opened a new log
I20260806 07:57:28.845515 2117 raft_consensus.cc:359] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } }
I20260806 07:57:28.845904 2117 raft_consensus.cc:385] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:28.846029 2117 raft_consensus.cc:740] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a5a432a06e4fc0a70244371d526dc4, State: Initialized, Role: FOLLOWER
I20260806 07:57:28.846648 2117 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } }
I20260806 07:57:28.847618 2126 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } } }
I20260806 07:57:28.848026 2126 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:28.848704 2117 sys_catalog.cc:565] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:57:28.859226 2137 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:57:28.859407 2137 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:57:28.867847 2049 master_runner.cc:429] Detected that this master f5a5a432a06e4fc0a70244371d526dc4 is joining an existing cluster
I20260806 07:57:28.868060 2049 catalog_manager.cc:1024] Shutting down the Catalog Manager
I20260806 07:57:28.869479 2049 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 07:57:28.869937 2049 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 07:57:28.870172 2049 tablet_replica.cc:333] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: stopping tablet replica
I20260806 07:57:28.872064 2049 master_runner.cc:434] Initiating AddMaster RPC to add 127.20.202.251:36615
Aug 06 07:57:28 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003045, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:57:28.970927 1646 catalog_manager.cc:7202] Initiating ChangeConfig request to add master 127.20.202.251:36615: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4"
member_type: NON_VOTER
last_known_addr {
host: "127.20.202.251"
port: 36615
}
attrs {
promote: true
}
}
dest_uuid: "66a12f0d487a4fa781a4cd6253ded947"
cas_config_opid_index: -1
I20260806 07:57:28.972674 1646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } }
I20260806 07:57:28.976095 1733 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 07:57:28.977248 1812 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20260806 07:57:28.977290 1629 consensus_peers.cc:597] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 -> Peer f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251:36615): Couldn't send request to peer f5a5a432a06e4fc0a70244371d526dc4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:57:28.977867 1905 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:57:28.978220 2150 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 07:57:28.992827 1905 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:28.993763 1811 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.005281 2154 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.006148 2154 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:29.007189 1905 catalog_manager.cc:7206] Successfully completed master ChangeConfig request to add master 127.20.202.251:36615
I20260806 07:57:29.008047 2049 master.cc:562] Master@127.20.202.251:36615 shutting down...
I20260806 07:57:29.009565 2150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.009874 2150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:29.010099 1905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.010355 1905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:29.010205 2153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.010507 2153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:29.010143 1734 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.020433 2157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.020839 2157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:29.148108 2036 heartbeater.cc:499] Master 127.20.202.254:35569 was elected leader, sending a full tablet report...
I20260806 07:57:29.346472 2049 master.cc:584] Master@127.20.202.251:36615 shutdown complete.
I20260806 07:57:29.346556 2049 master_runner.cc:306] Clearing existing system tablet
I20260806 07:57:29.346984 2049 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:29.347066 2049 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:29.349383 2049 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 07:57:29.350915 2160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:29.351307 2049 fs_report.cc:389] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:29.351536 2049 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 07:57:29.351675 2049 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "f5a5a432a06e4fc0a70244371d526dc4"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:29.354877 2049 ts_tablet_manager.cc:1916] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:57:29.359776 2049 ts_tablet_manager.cc:1929] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 07:57:29.359959 2049 log.cc:1199] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal/wals/00000000000000000000000000000000
I20260806 07:57:29.360349 2049 ts_tablet_manager.cc:1950] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Deleting consensus metadata
I20260806 07:57:29.360606 2049 master_runner.cc:316] Copying system tablet from 127.20.202.254:35569
I20260806 07:57:29.363137 2049 tablet_copy_client.cc:323] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: tablet copy: Beginning tablet copy session from remote peer at address 127.20.202.254:35569
I20260806 07:57:29.375028 1667 tablet_copy_service.cc:140] P 66a12f0d487a4fa781a4cd6253ded947: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f5a5a432a06e4fc0a70244371d526dc4 ({username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:58394)
I20260806 07:57:29.375370 1667 tablet_copy_service.cc:161] P 66a12f0d487a4fa781a4cd6253ded947: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f5a5a432a06e4fc0a70244371d526dc4 at {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:58394: session id = f5a5a432a06e4fc0a70244371d526dc4-00000000000000000000000000000000
I20260806 07:57:29.378319 1667 tablet_copy_source_session.cc:215] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 07:57:29.380576 2049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 07:57:29.385942 2049 tablet_copy_client.cc:806] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: tablet copy: Starting download of 0 data blocks...
I20260806 07:57:29.386229 2049 tablet_copy_client.cc:670] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: tablet copy: Starting download of 1 WAL segments...
I20260806 07:57:29.389302 2049 tablet_copy_client.cc:538] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 07:57:29.393255 1667 tablet_copy_service.cc:342] P 66a12f0d487a4fa781a4cd6253ded947: Request end of tablet copy session f5a5a432a06e4fc0a70244371d526dc4-00000000000000000000000000000000 received from {username='kudu', principal='kudu/127.20.202.251@KRBTEST.COM'} at 127.0.0.1:58394
I20260806 07:57:29.393472 1667 tablet_copy_service.cc:434] P 66a12f0d487a4fa781a4cd6253ded947: ending tablet copy session f5a5a432a06e4fc0a70244371d526dc4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f5a5a432a06e4fc0a70244371d526dc4
W20260806 07:57:29.400966 2049 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20260806 07:57:29.404734 2049 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:29.407227 2167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:29.407968 2168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:29.408784 2170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:29.414788 2049 server_base.cc:1061] running on GCE node
I20260806 07:57:29.415071 2049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:29.416023 2049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:29.417249 2049 hybrid_clock.cc:648] HybridClock initialized: now 1786003049417148 us; error 80 us; skew 500 ppm
I20260806 07:57:29.417484 2049 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:29 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003049, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.251@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:29.419401 2049 init.cc:377] Logged in from keytab as kudu/127.20.202.251@KRBTEST.COM (short username kudu)
I20260806 07:57:29.420682 2049 webserver.cc:533] Webserver started at http://127.20.202.251:45667/ using document root <none> and password file <none>
I20260806 07:57:29.420993 2049 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:29.421113 2049 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:29.423404 2049 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 07:57:29.425117 2177 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:29.425576 2049 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 07:57:29.425812 2049 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
uuid: "f5a5a432a06e4fc0a70244371d526dc4"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:29.426024 2049 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:29.435176 2049 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:29.438148 2049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:29.457574 2049 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.251:36615
I20260806 07:57:29.457767 2229 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.251:36615 every 8 connection(s)
I20260806 07:57:29.458549 2049 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-3/data/info.pb
I20260806 07:57:29.462376 2230 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:57:29.465040 2230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Bootstrap starting.
I20260806 07:57:29.477763 2230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:57:29.478183 2230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: Bootstrap complete.
I20260806 07:57:29.478969 2230 raft_consensus.cc:359] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } }
I20260806 07:57:29.479159 2230 raft_consensus.cc:740] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f5a5a432a06e4fc0a70244371d526dc4, State: Initialized, Role: LEARNER
I20260806 07:57:29.479429 2230 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } }
I20260806 07:57:29.480315 2230 sys_catalog.cc:565] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:29.480244 2234 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.480520 2234 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:57:29.487773 2245 catalog_manager.cc:1295] Loaded cluster ID: aadeb987aefa4f2b8774300d97344b0e
I20260806 07:57:29.487887 2245 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: loading cluster ID for follower catalog manager: success
I20260806 07:57:29.490420 2245 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: acquiring CA information for follower catalog manager: success
I20260806 07:57:29.492434 2245 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:29.498908 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1904
Aug 06 07:57:29 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:29.515266 2030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:29.558461 2205 raft_consensus.cc:1217] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20260806 07:57:29.559229 2234 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a12f0d487a4fa781a4cd6253ded947. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:29.559477 2234 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: This master's current role is: LEARNER
I20260806 07:57:29.605310 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1904
I20260806 07:57:29.624732 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37707
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--webserver_port=43637
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:29.982662 2249 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:29.983083 2249 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:29.983263 2249 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:29.992020 2249 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:29.992264 2249 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:30.003284 2249 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43637
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:30.005076 2249 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:30.006660 2249 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:30.017176 2254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:30.017201 2255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:30.018352 2257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:30.019956 2249 server_base.cc:1061] running on GCE node
I20260806 07:57:30.020507 2249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:30.021731 2249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:30.025094 2249 hybrid_clock.cc:648] HybridClock initialized: now 1786003050024931 us; error 126 us; skew 500 ppm
I20260806 07:57:30.025408 2249 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:30 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:30.029549 2249 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:30.031543 2249 webserver.cc:533] Webserver started at http://127.20.202.193:43637/ using document root <none> and password file <none>
I20260806 07:57:30.032401 2249 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:30.032594 2249 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:30.039359 2249 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:57:30.042676 2264 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:30.044354 2249 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:57:30.044591 2249 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:30.045454 2249 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:30.067711 2249 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:30.072372 2249 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:30.072738 2249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:30.073997 2249 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:30.075937 2249 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:30.076104 2249 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:30.076246 2249 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:30.076344 2249 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:30.116398 2249 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37707
I20260806 07:57:30.116482 2378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37707 every 8 connection(s)
I20260806 07:57:30.118245 2249 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:30.128697 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2249
Aug 06 07:57:30 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:57:30 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
Aug 06 07:57:30 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:30 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
I20260806 07:57:30.160010 2382 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38203
I20260806 07:57:30.160511 2382 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:30.161715 2382 heartbeater.cc:507] Master 127.20.202.253:38203 requested a full tablet report, sending...
I20260806 07:57:30.162415 2381 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35569
I20260806 07:57:30.162577 2381 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:30.162910 2380 heartbeater.cc:344] Connected to a master server at 127.20.202.252:44527
I20260806 07:57:30.163132 2380 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:30.163201 2381 heartbeater.cc:507] Master 127.20.202.254:35569 requested a full tablet report, sending...
I20260806 07:57:30.163617 2379 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36615
I20260806 07:57:30.163707 2380 heartbeater.cc:507] Master 127.20.202.252:44527 requested a full tablet report, sending...
I20260806 07:57:30.163743 2379 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:30.164162 2379 heartbeater.cc:507] Master 127.20.202.251:36615 requested a full tablet report, sending...
I20260806 07:57:30.164186 2150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: NON_VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: true } } }
I20260806 07:57:30.164455 2150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:30.164810 1723 ts_manager.cc:194] Re-registered known tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:30.165396 1646 ts_manager.cc:194] Re-registered known tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:30.165834 2150 raft_consensus.cc:1064] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: attempting to promote NON_VOTER f5a5a432a06e4fc0a70244371d526dc4 to VOTER
I20260806 07:57:30.166481 1801 ts_manager.cc:194] Re-registered known tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:30.166590 1646 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:34893
I20260806 07:57:30.166865 2194 ts_manager.cc:194] Registered new tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:30.166958 2150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:30.170271 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:57:30.170764 2205 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 LEARNER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:57:30.170892 1811 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:57:30.171979 2397 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:30.172922 2392 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:30.173205 1734 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Refusing update from remote peer 66a12f0d487a4fa781a4cd6253ded947: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 07:57:30.173785 2392 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:30.180182 1734 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:30.180126 2396 raft_consensus.cc:2955] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:30.181864 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:57:30.182072 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1613
I20260806 07:57:30.183334 1677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:30.184962 2402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:30.185353 2402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:30.308146 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1613
I20260806 07:57:30.307788 1811 raft_consensus.cc:2955] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:30.313337 2403 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a12f0d487a4fa781a4cd6253ded947" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:30.313771 2403 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:57:30.333369 1788 connection.cc:573] server connection from 127.0.0.1:43388 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 07:57:30.334316 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35569
--webserver_interface=127.20.202.254
--webserver_port=44175
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:30.694370 2404 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:30.694676 2404 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:30.694762 2404 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:30.703231 2404 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:30.703336 2404 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:30.703392 2404 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:30.703449 2404 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:30.714438 2404 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:35569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=44175
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:30.716081 2404 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:30.717566 2404 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:30.726295 2410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:30.726562 2409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:30.727496 2404 server_base.cc:1061] running on GCE node
W20260806 07:57:30.727609 2412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:30.728490 2404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:30.730225 2404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:30.731464 2404 hybrid_clock.cc:648] HybridClock initialized: now 1786003050731320 us; error 112 us; skew 500 ppm
I20260806 07:57:30.731834 2404 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:30 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:30.735276 2404 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:30.737028 2404 webserver.cc:533] Webserver started at http://127.20.202.254:44175/ using document root <none> and password file <none>
I20260806 07:57:30.737851 2404 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:30.738005 2404 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:30.743700 2404 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:57:30.746619 2419 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:30.748171 2404 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:57:30.748332 2404 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "66a12f0d487a4fa781a4cd6253ded947"
format_stamp: "Formatted at 2026-08-06 07:57:25 on dist-test-slave-g350"
I20260806 07:57:30.749047 2404 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:30.782337 2404 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:30.790756 2404 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:30.791064 2404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:30.812299 2404 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:35569
I20260806 07:57:30.812316 2471 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:35569 every 8 connection(s)
I20260806 07:57:30.814081 2404 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:30.818569 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2404
I20260806 07:57:30.818902 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2249
I20260806 07:57:30.822602 2472 sys_catalog.cc:263] Verifying existing consensus state
I20260806 07:57:30.827242 2472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Bootstrap starting.
I20260806 07:57:30.836426 2374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:30.848513 2472 log.cc:826] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:30.859169 2472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 07:57:30.859750 2472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: Bootstrap complete.
I20260806 07:57:30.867027 2472 raft_consensus.cc:359] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:30.867724 2472 raft_consensus.cc:740] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a12f0d487a4fa781a4cd6253ded947, State: Initialized, Role: FOLLOWER
I20260806 07:57:30.868547 2472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:30.869526 2478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:30.869926 2478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:30.870708 2472 sys_catalog.cc:565] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:30.881942 2489 catalog_manager.cc:1295] Loaded cluster ID: aadeb987aefa4f2b8774300d97344b0e
I20260806 07:57:30.882108 2489 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: loading cluster ID for follower catalog manager: success
I20260806 07:57:30.888125 2489 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: acquiring CA information for follower catalog manager: success
I20260806 07:57:30.891397 2489 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 07:57:30.955370 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2249
I20260806 07:57:30.955753 2404 master_runner.cc:241] Existing masters have pending config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:30.956142 2404 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:57:30.975903 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37707
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--webserver_port=43637
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:31.377738 2496 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:31.378082 2496 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:31.378239 2496 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:31.386663 2496 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:31.386965 2496 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:31.397949 2496 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43637
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:31.399655 2496 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:31.401093 2496 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:31.412134 2502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:31.412485 2501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:31.412624 2504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:31.413260 2496 server_base.cc:1061] running on GCE node
I20260806 07:57:31.413897 2496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:31.415017 2496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:31.416277 2496 hybrid_clock.cc:648] HybridClock initialized: now 1786003051416104 us; error 131 us; skew 500 ppm
I20260806 07:57:31.416651 2496 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:31 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:31.419935 2496 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:31.421708 2496 webserver.cc:533] Webserver started at http://127.20.202.193:43637/ using document root <none> and password file <none>
I20260806 07:57:31.422447 2496 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:31.422598 2496 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:31.428308 2496 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 07:57:31.431262 2512 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:31.432735 2496 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:57:31.432919 2496 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:31.433740 2496 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:31.451349 2496 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:31.455425 2496 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:31.455732 2496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:31.456687 2496 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:31.458722 2496 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:31.458837 2496 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:31.458945 2496 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:31.459029 2496 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:31.496071 2496 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37707
I20260806 07:57:31.496126 2624 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37707 every 8 connection(s)
I20260806 07:57:31.497742 2496 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:31.505684 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2496
Aug 06 07:57:31 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for hive/127.0.0.1@KRBTEST.COM
Aug 06 07:57:31 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.253@KRBTEST.COM
Aug 06 07:57:31 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.252@KRBTEST.COM
Aug 06 07:57:31 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.251@KRBTEST.COM
Aug 06 07:57:31 dist-test-slave-g350 krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:31.540126 2628 heartbeater.cc:344] Connected to a master server at 127.20.202.253:38203
I20260806 07:57:31.540560 2628 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:31.541769 2628 heartbeater.cc:507] Master 127.20.202.253:38203 requested a full tablet report, sending...
I20260806 07:57:31.542207 2626 heartbeater.cc:344] Connected to a master server at 127.20.202.252:44527
I20260806 07:57:31.542374 2626 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:31.542961 2626 heartbeater.cc:507] Master 127.20.202.252:44527 requested a full tablet report, sending...
I20260806 07:57:31.544021 2627 heartbeater.cc:344] Connected to a master server at 127.20.202.254:35569
I20260806 07:57:31.544153 2627 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:31.544166 1723 ts_manager.cc:194] Re-registered known tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:31.544679 2627 heartbeater.cc:507] Master 127.20.202.254:35569 requested a full tablet report, sending...
I20260806 07:57:31.545341 1801 ts_manager.cc:194] Re-registered known tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:31.546818 2437 ts_manager.cc:194] Registered new tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:31.550277 2625 heartbeater.cc:344] Connected to a master server at 127.20.202.251:36615
I20260806 07:57:31.550411 2625 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:31.550906 2625 heartbeater.cc:507] Master 127.20.202.251:36615 requested a full tablet report, sending...
I20260806 07:57:31.552197 2194 ts_manager.cc:194] Re-registered known tserver with Master: a52d3b09246d4d40af67e2d2676eff66 (127.20.202.193:37707)
I20260806 07:57:31.658934 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2496
W20260806 07:57:31.660203 2642 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:31.656358 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.20.202.193:37707)
0806 07:57:31.657041 (+ 683us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:31.657050 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:31.657090 (+ 40us) server_negotiation.cc:408] Connection header received
0806 07:57:31.657246 (+ 156us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:31.657254 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:31.657490 (+ 236us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:31.657771 (+ 281us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:31.658773 (+ 1002us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38444: BlockingRecv error: recv got EOF from 127.0.0.1:38444 (error 108)
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":319,"threads_started":1}
W20260806 07:57:31.661319 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2496: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:37707: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:57:31.661425 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2496
I20260806 07:57:31.680186 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37707
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--webserver_port=43637
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--fault_before_start=1.0 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:31.681463 2644 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 66a12f0d487a4fa781a4cd6253ded947)
I20260806 07:57:31.681797 2644 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:31.684749 1811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f6862cc5b8541188d81f82d5bd845da" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" is_pre_election: true
I20260806 07:57:31.685686 2644 leader_election.cc:290] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66a12f0d487a4fa781a4cd6253ded947 (127.20.202.254:35569), b9a97a8d315d4084b8fe58d9a0d0971b (127.20.202.252:44527), f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251:36615)
I20260806 07:57:31.694350 2447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f6862cc5b8541188d81f82d5bd845da" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "66a12f0d487a4fa781a4cd6253ded947" is_pre_election: true
I20260806 07:57:31.695004 2447 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f6862cc5b8541188d81f82d5bd845da in term 1.
I20260806 07:57:31.696386 2205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f6862cc5b8541188d81f82d5bd845da" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "f5a5a432a06e4fc0a70244371d526dc4" is_pre_election: true
I20260806 07:57:31.696455 2649 raft_consensus.cc:493] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 66a12f0d487a4fa781a4cd6253ded947)
I20260806 07:57:31.696734 2649 raft_consensus.cc:515] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:31.697445 2205 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f6862cc5b8541188d81f82d5bd845da in term 1.
I20260806 07:57:31.698004 1707 leader_election.cc:304] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 66a12f0d487a4fa781a4cd6253ded947, 6f6862cc5b8541188d81f82d5bd845da, f5a5a432a06e4fc0a70244371d526dc4; no voters: b9a97a8d315d4084b8fe58d9a0d0971b
I20260806 07:57:31.698427 2644 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:57:31.698803 2644 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 66a12f0d487a4fa781a4cd6253ded947)
I20260806 07:57:31.698987 2644 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:57:31.702430 2644 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 2 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:31.703707 2447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f6862cc5b8541188d81f82d5bd845da" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "66a12f0d487a4fa781a4cd6253ded947"
I20260806 07:57:31.703827 2644 leader_election.cc:290] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [CANDIDATE]: Term 2 election: Requested vote from peers 66a12f0d487a4fa781a4cd6253ded947 (127.20.202.254:35569), b9a97a8d315d4084b8fe58d9a0d0971b (127.20.202.252:44527), f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251:36615)
I20260806 07:57:31.703971 2447 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:57:31.704463 1811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f6862cc5b8541188d81f82d5bd845da" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b"
I20260806 07:57:31.704667 2205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f6862cc5b8541188d81f82d5bd845da" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "f5a5a432a06e4fc0a70244371d526dc4"
I20260806 07:57:31.704845 2205 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:57:31.705899 2649 leader_election.cc:290] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66a12f0d487a4fa781a4cd6253ded947 (127.20.202.254:35569), 6f6862cc5b8541188d81f82d5bd845da (127.20.202.253:38203), b9a97a8d315d4084b8fe58d9a0d0971b (127.20.202.252:44527)
I20260806 07:57:31.708412 2205 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f6862cc5b8541188d81f82d5bd845da in term 2.
I20260806 07:57:31.711138 2447 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f6862cc5b8541188d81f82d5bd845da in term 2.
I20260806 07:57:31.711645 1709 leader_election.cc:304] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 66a12f0d487a4fa781a4cd6253ded947, 6f6862cc5b8541188d81f82d5bd845da, f5a5a432a06e4fc0a70244371d526dc4; no voters: b9a97a8d315d4084b8fe58d9a0d0971b
I20260806 07:57:31.712958 2644 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:57:31.714214 2447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5a5a432a06e4fc0a70244371d526dc4" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "66a12f0d487a4fa781a4cd6253ded947" is_pre_election: true
I20260806 07:57:31.714609 2447 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5a5a432a06e4fc0a70244371d526dc4 in current term 2: Already voted for candidate 6f6862cc5b8541188d81f82d5bd845da in this term.
I20260806 07:57:31.725507 1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5a5a432a06e4fc0a70244371d526dc4" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "6f6862cc5b8541188d81f82d5bd845da" is_pre_election: true
I20260806 07:57:31.725776 1811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5a5a432a06e4fc0a70244371d526dc4" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" is_pre_election: true
I20260806 07:57:31.726249 2644 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [term 2 LEADER]: Becoming Leader. State: Replica: 6f6862cc5b8541188d81f82d5bd845da, State: Running, Role: LEADER
I20260806 07:57:31.726444 2178 leader_election.cc:304] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5a5a432a06e4fc0a70244371d526dc4; no voters: 66a12f0d487a4fa781a4cd6253ded947, b9a97a8d315d4084b8fe58d9a0d0971b
I20260806 07:57:31.726948 2649 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:57:31.726890 2644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:31.729455 2653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f6862cc5b8541188d81f82d5bd845da. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:31.729848 2653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:31.730576 2655 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:31.732484 2655 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:31.733374 2655 catalog_manager.cc:1295] Loaded cluster ID: aadeb987aefa4f2b8774300d97344b0e
I20260806 07:57:31.733443 2655 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:31.734755 2655 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:31.735563 2655 catalog_manager.cc:6081] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da: Loaded TSK: 0
I20260806 07:57:31.736493 2655 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:57:31.808640 2656 raft_consensus.cc:493] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 66a12f0d487a4fa781a4cd6253ded947)
I20260806 07:57:31.808876 2656 raft_consensus.cc:515] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } }
I20260806 07:57:31.810850 2656 leader_election.cc:290] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66a12f0d487a4fa781a4cd6253ded947 (127.20.202.254:35569), 6f6862cc5b8541188d81f82d5bd845da (127.20.202.253:38203), f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251:36615)
I20260806 07:57:31.811950 1734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "6f6862cc5b8541188d81f82d5bd845da" is_pre_election: true
I20260806 07:57:31.819211 2447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "66a12f0d487a4fa781a4cd6253ded947" is_pre_election: true
I20260806 07:57:31.819558 2447 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b9a97a8d315d4084b8fe58d9a0d0971b in current term 2: Already voted for candidate 6f6862cc5b8541188d81f82d5bd845da in this term.
I20260806 07:57:31.820936 1788 leader_election.cc:304] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b9a97a8d315d4084b8fe58d9a0d0971b; no voters: 66a12f0d487a4fa781a4cd6253ded947, 6f6862cc5b8541188d81f82d5bd845da
I20260806 07:57:31.821496 2656 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:57:31.824738 2656 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:57:31.830058 2205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "f5a5a432a06e4fc0a70244371d526dc4" is_pre_election: true
I20260806 07:57:31.830391 2205 raft_consensus.cc:2393] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b9a97a8d315d4084b8fe58d9a0d0971b in current term 2: Already voted for candidate 6f6862cc5b8541188d81f82d5bd845da in this term.
W20260806 07:57:32.077153 2648 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:32.077459 2648 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:32.077551 2648 flags.cc:432] Enabled unsafe flag: --fault_before_start=1
W20260806 07:57:32.077668 2648 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:32.086376 2648 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:32.086581 2648 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:32.097262 2648 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43637
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--fault_before_start=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:32.098937 2648 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:32.100279 2648 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:32.110332 2663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:32.110251 2664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:32.110756 2666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:32.111300 2648 server_base.cc:1061] running on GCE node
I20260806 07:57:32.112025 2648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:32.113302 2648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:32.113296 1811 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [term 2 FOLLOWER]: Refusing update from remote peer 6f6862cc5b8541188d81f82d5bd845da: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:57:32.114131 2653 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:57:32.114821 2648 hybrid_clock.cc:648] HybridClock initialized: now 1786003052114599 us; error 169 us; skew 500 ppm
I20260806 07:57:32.115139 2648 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:57:32.116621 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f6862cc5b8541188d81f82d5bd845da. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.116917 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 07:57:32 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003052, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:32.119042 2648 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:32.120787 2648 webserver.cc:533] Webserver started at http://127.20.202.193:43637/ using document root <none> and password file <none>
I20260806 07:57:32.121580 2648 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:32.121760 2648 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:32.127738 2648 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 07:57:32.131017 2674 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:32.132520 2648 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:57:32.132725 2648 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:32.133513 2648 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:32.180711 2648 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:32.185012 2648 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:32.185321 2648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:32.186430 2648 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:32.188045 2648 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:32.188166 2648 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:32.188318 2648 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:32.188392 2648 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 07:57:32.188782 2648 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260806 07:57:32.188879 2648 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260806 07:57:32.188964 2648 fault_injection.cc:59] injecting fault for kudu (pid 2648): FLAGS_fault_before_start (process will exit)
I20260806 07:57:32.208303 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:37707
--local_ip_for_outbound_sockets=127.20.202.193
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527
--webserver_port=43637
--webserver_interface=127.20.202.193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615 with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 07:57:32.218195 2205 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 2 FOLLOWER]: Refusing update from remote peer 6f6862cc5b8541188d81f82d5bd845da: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:57:32.218922 2653 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 07:57:32.220373 2205 raft_consensus.cc:2955] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [term 2 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f5a5a432a06e4fc0a70244371d526dc4 (127.20.202.251) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.225214 2684 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.225304 2649 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f6862cc5b8541188d81f82d5bd845da. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.225610 2684 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:32.225656 2649 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:32.229755 2447 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [term 2 FOLLOWER]: Refusing update from remote peer 6f6862cc5b8541188d81f82d5bd845da: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20260806 07:57:32.230876 2653 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 07:57:32.237530 2649 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.237829 2649 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5a432a06e4fc0a70244371d526dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:32.239218 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.239343 2644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.239531 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a97a8d315d4084b8fe58d9a0d0971b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:32.239570 2644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:32.240001 2686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.240317 2686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:32.243675 2694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.244124 2694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:57:32.244495 2689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f6862cc5b8541188d81f82d5bd845da. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.244724 2689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a12f0d487a4fa781a4cd6253ded947 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 07:57:32.569134 2682 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:32.569475 2682 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:32.569686 2682 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:32.578148 2682 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:32.578383 2682 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:32.589136 2682 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36687,127.20.202.213:43507,127.20.202.214:40961,127.20.202.215:34767,127.20.202.216:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:37707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=43637
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:35569,127.20.202.253:38203,127.20.202.252:44527,127.20.202.251:36615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:32.590894 2682 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:32.592295 2682 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:32.602540 2700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:32.603219 2702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:32.603238 2699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:32.603756 2682 server_base.cc:1061] running on GCE node
I20260806 07:57:32.604310 2682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:32.605386 2682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:32.606726 2682 hybrid_clock.cc:648] HybridClock initialized: now 1786003052606544 us; error 149 us; skew 500 ppm
I20260806 07:57:32.607074 2682 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:32 dist-test-slave-g350 krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003052, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:32.610693 2682 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:32.612459 2682 webserver.cc:533] Webserver started at http://127.20.202.193:43637/ using document root <none> and password file <none>
I20260806 07:57:32.613214 2682 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:32.613363 2682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:32.619588 2682 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 07:57:32.622547 2709 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:32.624090 2682 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 07:57:32.624293 2682 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "a52d3b09246d4d40af67e2d2676eff66"
format_stamp: "Formatted at 2026-08-06 07:57:28 on dist-test-slave-g350"
I20260806 07:57:32.625023 2682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:32.659019 2682 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:32.662979 2682 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:32.663273 2682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:32.664289 2682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:32.665934 2682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:32.666061 2682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:32.666195 2682 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:32.666285 2682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:32.703341 2682 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37707
I20260806 07:57:32.703393 2822 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37707 every 8 connection(s)
I20260806 07:57:32.704913 2682 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestAddMaster_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:32.707883 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2682
I20260806 07:57:32.708244 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2682
W20260806 07:57:32.734165 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2682: Not authorized: Client connection negotiation failed: client connection to 127.20.202.193:37707: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:57:32.734380 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2682
I20260806 07:57:32.740820 2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6f6862cc5b8541188d81f82d5bd845da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "66a12f0d487a4fa781a4cd6253ded947" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 35569 } } peers { permanent_uuid: "6f6862cc5b8541188d81f82d5bd845da" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 38203 } } peers { permanent_uuid: "b9a97a8d315d4084b8fe58d9a0d0971b" member_type: VOTER last_known_addr { host: "127.20.202.252" port: 44527 } } peers { permanent_uuid: "f5a5a432a06e4fc0a70244371d526dc4" member_type: VOTER last_known_addr { host: "127.20.202.251" port: 36615 } attrs { promote: false } } }
I20260806 07:57:32.741109 2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6862cc5b8541188d81f82d5bd845da [sys.catalog]: This master's current role is: LEADER
W20260806 07:57:32.754734 2837 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:32.723198 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:34411 (local address 127.20.202.252:44527)
0806 07:57:32.736641 (+ 13443us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:32.736648 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:32.753718 (+ 17070us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:34411: BlockingRecv error: recv got EOF from 127.20.202.193:34411 (error 108)
Metrics: {"server-negotiator.queue_time_us":13302,"thread_start_us":13197,"threads_started":1}
W20260806 07:57:32.754771 2827 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:32.709185 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:47971 (local address 127.20.202.251:36615)
0806 07:57:32.718365 (+ 9180us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:32.718375 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:32.722903 (+ 4528us) server_negotiation.cc:408] Connection header received
0806 07:57:32.729952 (+ 7049us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:32.729960 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:32.730042 (+ 82us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:32.730184 (+ 142us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:32.753777 (+ 23593us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:47971: BlockingRecv error: recv got EOF from 127.20.202.193:47971 (error 108)
Metrics: {"server-negotiator.queue_time_us":8991,"thread_start_us":9370,"threads_started":1}
W20260806 07:57:32.754781 2828 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:32.709186 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:40885 (local address 127.20.202.254:35569)
0806 07:57:32.722311 (+ 13125us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:32.722319 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:32.722341 (+ 22us) server_negotiation.cc:408] Connection header received
0806 07:57:32.724002 (+ 1661us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:32.724013 (+ 11us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:32.724127 (+ 114us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:32.724244 (+ 117us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:32.728133 (+ 3889us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:57:32.730440 (+ 2307us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0806 07:57:32.732414 (+ 1974us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0806 07:57:32.732970 (+ 556us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0806 07:57:32.753927 (+ 20957us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:40885: BlockingRecv error: recv got EOF from 127.20.202.193:40885 (error 108)
Metrics: {"server-negotiator.queue_time_us":12995,"thread_start_us":20554,"threads_started":1}
W20260806 07:57:32.754781 2835 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:32.722967 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.20.202.193:42207 (local address 127.20.202.253:38203)
0806 07:57:32.736778 (+ 13811us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:32.736783 (+ 5us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:32.736802 (+ 19us) server_negotiation.cc:408] Connection header received
0806 07:57:32.736853 (+ 51us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:32.736857 (+ 4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:32.736917 (+ 60us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:32.737027 (+ 110us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:32.753772 (+ 16745us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.202.193:42207: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":13694,"thread_start_us":14152,"threads_started":1}
I20260806 07:57:32.755692 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2404
W20260806 07:57:32.757851 21291 external_mini_cluster.cc:1721] Unable to check leaks on /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2404: Not authorized: Client connection negotiation failed: client connection to 127.20.202.254:35569: server requires authentication, but client does not have Kerberos credentials available
I20260806 07:57:32.758002 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2404
W20260806 07:57:32.758087 2828 negotiation.cc:336] Failed RPC negotiation. Trace:
0806 07:57:32.756575 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.20.202.254:35569)
0806 07:57:32.756752 (+ 177us) server_negotiation.cc:207] Beginning negotiation
0806 07:57:32.756756 (+ 4us) server_negotiation.cc:400] Waiting for connection header
0806 07:57:32.756774 (+ 18us) server_negotiation.cc:408] Connection header received
0806 07:57:32.756890 (+ 116us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0806 07:57:32.756895 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0806 07:57:32.756960 (+ 65us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0806 07:57:32.757055 (+ 95us) server_negotiation.cc:227] Negotiated authn=SASL
0806 07:57:32.757886 (+ 831us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58440: BlockingRecv error: recv got EOF from 127.0.0.1:58440 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 07:57:32.779964 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1691
I20260806 07:57:32.781198 1754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:32.897846 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1691
I20260806 07:57:32.921094 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 1769
I20260806 07:57:32.922322 1832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:33.038519 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 1769
I20260806 07:57:33.061252 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2049
I20260806 07:57:33.062520 2225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:33.181975 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2049
Shutting down hive metastore.
2026-08-06T07:57:33Z chronyd exiting
2026-08-06T07:57:33Z chronyd exiting
2026-08-06T07:57:33Z chronyd exiting
2026-08-06T07:57:33Z chronyd exiting
2026-08-06T07:57:33Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestAddMaster/11 (27517 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/0
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:33 dist-test-slave-g350 krb5kdc[2847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:33 dist-test-slave-g350 krb5kdc[2847](info): set up 2 sockets
Aug 06 07:57:33 dist-test-slave-g350 krb5kdc[2847](info): commencing operation
krb5kdc: starting...
W20260806 07:57:35.749240 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.082s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:35 dist-test-slave-g350 krb5kdc[2847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:35Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:36.141636 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44427
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:44427
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:36.507279 2863 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:36.507674 2863 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:36.507827 2863 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:36.516503 2863 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:36.516654 2863 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:36.516777 2863 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:36.516860 2863 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:36.527911 2863 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:44427
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:44427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:36.529613 2863 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:36.531008 2863 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:36.539664 2869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:36.539609 2868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:36.540161 2871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:36.541136 2863 server_base.cc:1061] running on GCE node
I20260806 07:57:36.541944 2863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:36.543427 2863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:36.544697 2863 hybrid_clock.cc:648] HybridClock initialized: now 1786003056544635 us; error 48 us; skew 500 ppm
I20260806 07:57:36.545128 2863 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:36 dist-test-slave-g350 krb5kdc[2847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003056, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:36.548533 2863 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:36.550366 2863 webserver.cc:533] Webserver started at http://127.20.202.254:36623/ using document root <none> and password file <none>
I20260806 07:57:36.551164 2863 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:36.551308 2863 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:36.551693 2863 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:36.554345 2863 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "2fc4f84617504106af3c53cbc6d713ff"
format_stamp: "Formatted at 2026-08-06 07:57:36 on dist-test-slave-g350"
I20260806 07:57:36.555145 2863 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "2fc4f84617504106af3c53cbc6d713ff"
format_stamp: "Formatted at 2026-08-06 07:57:36 on dist-test-slave-g350"
I20260806 07:57:36.560810 2863 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:57:36.564460 2878 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:36.566030 2863 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:36.566248 2863 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "2fc4f84617504106af3c53cbc6d713ff"
format_stamp: "Formatted at 2026-08-06 07:57:36 on dist-test-slave-g350"
I20260806 07:57:36.566481 2863 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:36.599004 2863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:36.602795 2863 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:36.603111 2863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:36.624809 2863 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:44427
I20260806 07:57:36.624845 2930 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:44427 every 8 connection(s)
I20260806 07:57:36.626605 2863 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:36.630878 2931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:36.634555 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2863
I20260806 07:57:36.634794 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:36.642215 2931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff: Bootstrap starting.
I20260806 07:57:36.646000 2931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:36.647405 2931 log.cc:826] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff: Log is configured to *not* fsync() on all Append() calls
Aug 06 07:57:36 dist-test-slave-g350 krb5kdc[2847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:36.650698 2931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff: No bootstrap required, opened a new log
I20260806 07:57:36.656152 2931 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc4f84617504106af3c53cbc6d713ff" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44427 } }
I20260806 07:57:36.656528 2931 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:36.656668 2931 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc4f84617504106af3c53cbc6d713ff, State: Initialized, Role: FOLLOWER
I20260806 07:57:36.657425 2931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc4f84617504106af3c53cbc6d713ff" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44427 } }
I20260806 07:57:36.657768 2931 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:57:36.657923 2931 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:57:36.658118 2931 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:36.660663 2931 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc4f84617504106af3c53cbc6d713ff" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44427 } }
I20260806 07:57:36.661248 2931 leader_election.cc:304] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fc4f84617504106af3c53cbc6d713ff; no voters:
I20260806 07:57:36.661999 2931 leader_election.cc:290] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:57:36.662189 2936 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:36.663043 2936 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [term 1 LEADER]: Becoming Leader. State: Replica: 2fc4f84617504106af3c53cbc6d713ff, State: Running, Role: LEADER
I20260806 07:57:36.663666 2936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc4f84617504106af3c53cbc6d713ff" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44427 } }
I20260806 07:57:36.664640 2931 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:36.667212 2937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fc4f84617504106af3c53cbc6d713ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc4f84617504106af3c53cbc6d713ff" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44427 } } }
I20260806 07:57:36.667186 2938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fc4f84617504106af3c53cbc6d713ff. Latest consensus state: current_term: 1 leader_uuid: "2fc4f84617504106af3c53cbc6d713ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc4f84617504106af3c53cbc6d713ff" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 44427 } } }
I20260806 07:57:36.667603 2937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:36.667603 2938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:36.668942 2942 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:36.676409 2942 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:36.687915 2942 catalog_manager.cc:1383] Generated new cluster ID: 237b7ad39ecc4251a9e9e0d9bc4e08bf
I20260806 07:57:36.688055 2942 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:36.702375 2942 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:36.703600 2942 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:36.719447 2942 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2fc4f84617504106af3c53cbc6d713ff: Generated new TSK 0
I20260806 07:57:36.720750 2942 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:36.861449 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:44427
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:36079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:37.219688 2959 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:37.220042 2959 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:37.220139 2959 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:37.228609 2959 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:37.228792 2959 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:37.239600 2959 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:36079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:44427
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:37.241185 2959 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:37.242671 2959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:37.252331 2964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:37.252434 2965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:37.253484 2967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:37.254235 2959 server_base.cc:1061] running on GCE node
I20260806 07:57:37.254951 2959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:37.256001 2959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:37.257272 2959 hybrid_clock.cc:648] HybridClock initialized: now 1786003057257201 us; error 58 us; skew 500 ppm
I20260806 07:57:37.257732 2959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[2847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003057, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:37.261047 2959 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:37.262809 2959 webserver.cc:533] Webserver started at http://127.20.202.193:42353/ using document root <none> and password file <none>
I20260806 07:57:37.263586 2959 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:37.263729 2959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:37.264084 2959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:37.266767 2959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "2fc3f3b1b6c047ff96f5499479aeba60"
format_stamp: "Formatted at 2026-08-06 07:57:37 on dist-test-slave-g350"
I20260806 07:57:37.267549 2959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "2fc3f3b1b6c047ff96f5499479aeba60"
format_stamp: "Formatted at 2026-08-06 07:57:37 on dist-test-slave-g350"
I20260806 07:57:37.273263 2959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:57:37.276742 2974 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:37.278131 2959 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:37.278344 2959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "2fc3f3b1b6c047ff96f5499479aeba60"
format_stamp: "Formatted at 2026-08-06 07:57:37 on dist-test-slave-g350"
I20260806 07:57:37.278550 2959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:37.294292 2959 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:37.298306 2959 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:37.298602 2959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:37.299561 2959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:37.301172 2959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:37.301277 2959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:37.301436 2959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:37.301510 2959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:37.340261 2959 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37695
I20260806 07:57:37.340325 3086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37695 every 8 connection(s)
I20260806 07:57:37.342218 2959 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:37.344524 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 2959
I20260806 07:57:37.344775 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_0.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[2847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003057, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:37.365486 3087 heartbeater.cc:344] Connected to a master server at 127.20.202.254:44427
I20260806 07:57:37.365978 3087 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:37.366977 3087 heartbeater.cc:507] Master 127.20.202.254:44427 requested a full tablet report, sending...
I20260806 07:57:37.369181 2895 ts_manager.cc:194] Registered new tserver with Master: 2fc3f3b1b6c047ff96f5499479aeba60 (127.20.202.193:37695)
I20260806 07:57:37.370887 2895 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:56503
I20260806 07:57:37.374661 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[2847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[2847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:57:37.442521 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2959
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[2847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:37.460397 3082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:37.547297 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2959
I20260806 07:57:37.566447 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 2863
I20260806 07:57:37.567832 2925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:37.676219 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 2863
2026-08-06T07:57:37Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/0 (4053 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/1
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[3101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[3101](info): set up 2 sockets
Aug 06 07:57:37 dist-test-slave-g350 krb5kdc[3101](info): commencing operation
krb5kdc: starting...
W20260806 07:57:39.834360 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.114s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:40 dist-test-slave-g350 krb5kdc[3101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:40Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:40.234256 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:38965
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:39771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:38965
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:40.582456 3117 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:40.582842 3117 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:40.583037 3117 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:40.591639 3117 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:40.591792 3117 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:40.591923 3117 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:40.592006 3117 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:40.602877 3117 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:38965
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:40.604627 3117 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:40.606106 3117 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:40.615371 3123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:40.615310 3124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:40.616197 3117 server_base.cc:1061] running on GCE node
W20260806 07:57:40.616935 3126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:40.617756 3117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:40.619180 3117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:40.620419 3117 hybrid_clock.cc:648] HybridClock initialized: now 1786003060620385 us; error 47 us; skew 500 ppm
I20260806 07:57:40.620772 3117 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:40 dist-test-slave-g350 krb5kdc[3101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:40.623912 3117 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:40.625461 3117 webserver.cc:533] Webserver started at http://127.20.202.254:34451/ using document root <none> and password file <none>
I20260806 07:57:40.626209 3117 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:40.626307 3117 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:40.626632 3117 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:40.628993 3117 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9"
format_stamp: "Formatted at 2026-08-06 07:57:40 on dist-test-slave-g350"
I20260806 07:57:40.629797 3117 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9"
format_stamp: "Formatted at 2026-08-06 07:57:40 on dist-test-slave-g350"
I20260806 07:57:40.635201 3117 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 07:57:40.638616 3133 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:40.639926 3117 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 07:57:40.640081 3117 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9"
format_stamp: "Formatted at 2026-08-06 07:57:40 on dist-test-slave-g350"
I20260806 07:57:40.640276 3117 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:40.657938 3117 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:40.661563 3117 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:40.661885 3117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:40.682916 3117 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:38965
I20260806 07:57:40.682953 3185 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:38965 every 8 connection(s)
I20260806 07:57:40.684474 3117 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:40.685832 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3117
I20260806 07:57:40.686010 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:40.689664 3186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 07:57:40 dist-test-slave-g350 krb5kdc[3101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:40.701701 3186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9: Bootstrap starting.
I20260806 07:57:40.705158 3186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:40.706279 3186 log.cc:826] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:40.709036 3186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9: No bootstrap required, opened a new log
I20260806 07:57:40.714627 3186 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38965 } }
I20260806 07:57:40.714998 3186 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:40.715116 3186 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3c76f8cb3b4d5fb5f47e80d624bde9, State: Initialized, Role: FOLLOWER
I20260806 07:57:40.715794 3186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38965 } }
I20260806 07:57:40.716018 3186 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:57:40.716156 3186 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:57:40.716331 3186 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:40.718945 3186 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38965 } }
I20260806 07:57:40.719434 3186 leader_election.cc:304] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3c76f8cb3b4d5fb5f47e80d624bde9; no voters:
I20260806 07:57:40.720032 3186 leader_election.cc:290] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:57:40.720185 3191 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:40.720883 3191 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [term 1 LEADER]: Becoming Leader. State: Replica: 7d3c76f8cb3b4d5fb5f47e80d624bde9, State: Running, Role: LEADER
I20260806 07:57:40.721462 3191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38965 } }
I20260806 07:57:40.722191 3186 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:40.724541 3193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38965 } } }
I20260806 07:57:40.724673 3194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3c76f8cb3b4d5fb5f47e80d624bde9. Latest consensus state: current_term: 1 leader_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3c76f8cb3b4d5fb5f47e80d624bde9" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38965 } } }
I20260806 07:57:40.724874 3193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:40.724924 3194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:40.727097 3198 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:40.733179 3198 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:40.743021 3198 catalog_manager.cc:1383] Generated new cluster ID: 40e311dcf8074cce923e0f117906be92
I20260806 07:57:40.743170 3198 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:40.753058 3198 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:40.754318 3198 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:40.772564 3198 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7d3c76f8cb3b4d5fb5f47e80d624bde9: Generated new TSK 0
I20260806 07:57:40.773528 3198 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:40.921343 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:38965
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:39771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:41.284734 3215 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:41.285121 3215 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:41.285333 3215 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:41.295288 3215 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:41.295555 3215 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:41.308156 3215 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:38965
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:41.310007 3215 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:41.311517 3215 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:41.322094 3220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:41.322027 3221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:41.324110 3223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:41.324435 3215 server_base.cc:1061] running on GCE node
I20260806 07:57:41.325083 3215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:41.326265 3215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:41.327524 3215 hybrid_clock.cc:648] HybridClock initialized: now 1786003061327492 us; error 45 us; skew 500 ppm
I20260806 07:57:41.327924 3215 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003061, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:41.331422 3215 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:41.333319 3215 webserver.cc:533] Webserver started at http://127.20.202.193:39115/ using document root <none> and password file <none>
I20260806 07:57:41.334158 3215 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:41.334265 3215 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:41.334618 3215 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:41.337242 3215 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "439bda00306a49d49ddfe14f9b7c97ff"
format_stamp: "Formatted at 2026-08-06 07:57:41 on dist-test-slave-g350"
I20260806 07:57:41.338099 3215 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "439bda00306a49d49ddfe14f9b7c97ff"
format_stamp: "Formatted at 2026-08-06 07:57:41 on dist-test-slave-g350"
I20260806 07:57:41.344094 3215 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 07:57:41.348217 3230 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:41.349814 3215 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:41.349967 3215 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "439bda00306a49d49ddfe14f9b7c97ff"
format_stamp: "Formatted at 2026-08-06 07:57:41 on dist-test-slave-g350"
I20260806 07:57:41.350159 3215 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:41.390038 3215 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:41.399957 3215 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:41.400259 3215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:41.401283 3215 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:41.403070 3215 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:41.403159 3215 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:41.403249 3215 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:41.403297 3215 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:41.441809 3215 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37667
I20260806 07:57:41.441830 3343 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37667 every 8 connection(s)
I20260806 07:57:41.443449 3215 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:41.445515 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3215
I20260806 07:57:41.445755 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_1.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003061, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:41.465060 3344 heartbeater.cc:344] Connected to a master server at 127.20.202.254:38965
I20260806 07:57:41.465386 3344 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:41.466270 3344 heartbeater.cc:507] Master 127.20.202.254:38965 requested a full tablet report, sending...
I20260806 07:57:41.468539 3151 ts_manager.cc:194] Registered new tserver with Master: 439bda00306a49d49ddfe14f9b7c97ff (127.20.202.193:37667)
I20260806 07:57:41.470290 3151 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:60645
I20260806 07:57:41.474596 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:57:41.537700 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3215
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:41.555825 3339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:41.643805 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3215
I20260806 07:57:41.664769 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3117
I20260806 07:57:41.666277 3181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:41.777940 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3117
2026-08-06T07:57:41Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/1 (4101 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/2
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3357](info): set up 2 sockets
Aug 06 07:57:41 dist-test-slave-g350 krb5kdc[3357](info): commencing operation
krb5kdc: starting...
W20260806 07:57:43.881248 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.059s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:44 dist-test-slave-g350 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:44Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:44.277370 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40403
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:40403
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:44.645905 3373 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:44.646292 3373 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:44.646466 3373 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:44.655030 3373 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:44.655139 3373 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:44.655241 3373 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:44.655329 3373 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:44.666699 3373 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:40403
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:40403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:44.668422 3373 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:44.669876 3373 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:44.678192 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:44.678398 3378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:44.678928 3381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:44.679675 3373 server_base.cc:1061] running on GCE node
I20260806 07:57:44.680404 3373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:44.681886 3373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:44.683081 3373 hybrid_clock.cc:648] HybridClock initialized: now 1786003064683012 us; error 57 us; skew 500 ppm
I20260806 07:57:44.683377 3373 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:44 dist-test-slave-g350 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003064, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:44.686545 3373 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:44.688105 3373 webserver.cc:533] Webserver started at http://127.20.202.254:39509/ using document root <none> and password file <none>
I20260806 07:57:44.688925 3373 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:44.689052 3373 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:44.689435 3373 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:44.691884 3373 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "29103e3e791c4b1bab30ecdbebb128af"
format_stamp: "Formatted at 2026-08-06 07:57:44 on dist-test-slave-g350"
I20260806 07:57:44.692682 3373 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "29103e3e791c4b1bab30ecdbebb128af"
format_stamp: "Formatted at 2026-08-06 07:57:44 on dist-test-slave-g350"
I20260806 07:57:44.698313 3373 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:57:44.701926 3388 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:44.703382 3373 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:57:44.703568 3373 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "29103e3e791c4b1bab30ecdbebb128af"
format_stamp: "Formatted at 2026-08-06 07:57:44 on dist-test-slave-g350"
I20260806 07:57:44.703789 3373 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:44.749068 3373 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:44.753546 3373 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:44.753907 3373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:44.775967 3440 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:40403 every 8 connection(s)
I20260806 07:57:44.775964 3373 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:40403
I20260806 07:57:44.777760 3373 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:44.780412 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3373
I20260806 07:57:44.780604 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:44.783118 3441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:44.793696 3441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af: Bootstrap starting.
Aug 06 07:57:44 dist-test-slave-g350 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:44.797438 3441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:44.798755 3441 log.cc:826] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:44.801692 3441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af: No bootstrap required, opened a new log
I20260806 07:57:44.807685 3441 raft_consensus.cc:359] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29103e3e791c4b1bab30ecdbebb128af" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40403 } }
I20260806 07:57:44.808051 3441 raft_consensus.cc:385] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:44.808192 3441 raft_consensus.cc:740] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29103e3e791c4b1bab30ecdbebb128af, State: Initialized, Role: FOLLOWER
I20260806 07:57:44.808828 3441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29103e3e791c4b1bab30ecdbebb128af" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40403 } }
I20260806 07:57:44.809082 3441 raft_consensus.cc:399] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:57:44.809271 3441 raft_consensus.cc:493] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:57:44.809435 3441 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:44.811981 3441 raft_consensus.cc:515] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29103e3e791c4b1bab30ecdbebb128af" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40403 } }
I20260806 07:57:44.812495 3441 leader_election.cc:304] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29103e3e791c4b1bab30ecdbebb128af; no voters:
I20260806 07:57:44.813154 3441 leader_election.cc:290] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:57:44.813315 3446 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:44.814168 3446 raft_consensus.cc:697] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [term 1 LEADER]: Becoming Leader. State: Replica: 29103e3e791c4b1bab30ecdbebb128af, State: Running, Role: LEADER
I20260806 07:57:44.814720 3446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29103e3e791c4b1bab30ecdbebb128af" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40403 } }
I20260806 07:57:44.815474 3441 sys_catalog.cc:565] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:44.818373 3447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29103e3e791c4b1bab30ecdbebb128af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29103e3e791c4b1bab30ecdbebb128af" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40403 } } }
I20260806 07:57:44.818472 3448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29103e3e791c4b1bab30ecdbebb128af. Latest consensus state: current_term: 1 leader_uuid: "29103e3e791c4b1bab30ecdbebb128af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29103e3e791c4b1bab30ecdbebb128af" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 40403 } } }
I20260806 07:57:44.818771 3447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:44.818796 3448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:44.820746 3454 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:44.828332 3454 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:44.838068 3454 catalog_manager.cc:1383] Generated new cluster ID: 9a7b1b0a05894b79a53970d5f6afa079
I20260806 07:57:44.838246 3454 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:44.849221 3454 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:44.850920 3454 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:44.866727 3454 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 29103e3e791c4b1bab30ecdbebb128af: Generated new TSK 0
I20260806 07:57:44.867666 3454 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:45.002424 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:40403
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:45.368001 3469 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:45.368371 3469 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:45.368541 3469 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:45.377225 3469 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:45.377466 3469 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:45.388286 3469 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:40403
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:45.389979 3469 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:45.391397 3469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:45.401743 3474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:45.401947 3475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:45.402665 3477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:45.403541 3469 server_base.cc:1061] running on GCE node
I20260806 07:57:45.404179 3469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:45.405179 3469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:45.406462 3469 hybrid_clock.cc:648] HybridClock initialized: now 1786003065406416 us; error 55 us; skew 500 ppm
I20260806 07:57:45.406859 3469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003065, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:45.410375 3469 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:45.412122 3469 webserver.cc:533] Webserver started at http://127.20.202.193:38409/ using document root <none> and password file <none>
I20260806 07:57:45.412918 3469 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:45.413067 3469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:45.413434 3469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:45.416059 3469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "3133278bc2af4512af7cebb9ad219085"
format_stamp: "Formatted at 2026-08-06 07:57:45 on dist-test-slave-g350"
I20260806 07:57:45.416837 3469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "3133278bc2af4512af7cebb9ad219085"
format_stamp: "Formatted at 2026-08-06 07:57:45 on dist-test-slave-g350"
I20260806 07:57:45.422646 3469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 07:57:45.426450 3484 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:45.427927 3469 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:57:45.428125 3469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "3133278bc2af4512af7cebb9ad219085"
format_stamp: "Formatted at 2026-08-06 07:57:45 on dist-test-slave-g350"
I20260806 07:57:45.428387 3469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:45.447626 3469 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:45.451514 3469 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:45.451826 3469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:45.452819 3469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:45.454538 3469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:45.454648 3469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:45.454813 3469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:45.454885 3469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:45.490370 3469 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:42263
I20260806 07:57:45.490418 3597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:42263 every 8 connection(s)
I20260806 07:57:45.491809 3469 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:45.494524 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3469
I20260806 07:57:45.494773 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_2.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003065, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:45.513314 3598 heartbeater.cc:344] Connected to a master server at 127.20.202.254:40403
I20260806 07:57:45.513798 3598 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:45.514618 3598 heartbeater.cc:507] Master 127.20.202.254:40403 requested a full tablet report, sending...
I20260806 07:57:45.516719 3405 ts_manager.cc:194] Registered new tserver with Master: 3133278bc2af4512af7cebb9ad219085 (127.20.202.193:42263)
I20260806 07:57:45.518373 3405 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:39349
I20260806 07:57:45.523321 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:57:45.587332 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3469
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:45.604421 3593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:45.689715 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3469
I20260806 07:57:45.708496 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3373
I20260806 07:57:45.709867 3436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:45.818260 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3373
2026-08-06T07:57:45Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/2 (4040 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/3
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3611](info): set up 2 sockets
Aug 06 07:57:45 dist-test-slave-g350 krb5kdc[3611](info): commencing operation
krb5kdc: starting...
W20260806 07:57:47.941309 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.079s user 0.004s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:48 dist-test-slave-g350 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:48.354257 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33853
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:33619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:33853
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:48.712179 3627 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:48.712502 3627 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:48.712682 3627 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:48.721238 3627 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:48.721340 3627 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:48.721436 3627 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:48.721529 3627 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:48.732478 3627 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:33619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:33853
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:33853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:48.734117 3627 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:48.735479 3627 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:48.744119 3633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:48.744122 3632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:48.745436 3635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:48.745821 3627 server_base.cc:1061] running on GCE node
I20260806 07:57:48.746759 3627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:48.748231 3627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:48.749457 3627 hybrid_clock.cc:648] HybridClock initialized: now 1786003068749401 us; error 51 us; skew 500 ppm
I20260806 07:57:48.749846 3627 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:48 dist-test-slave-g350 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003068, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:48.753146 3627 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:48.754869 3627 webserver.cc:533] Webserver started at http://127.20.202.254:38491/ using document root <none> and password file <none>
I20260806 07:57:48.755693 3627 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:48.755820 3627 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:48.756295 3627 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:48.758901 3627 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "70163fa1ed6846ff81682c8369ce79a1"
format_stamp: "Formatted at 2026-08-06 07:57:48 on dist-test-slave-g350"
I20260806 07:57:48.759755 3627 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "70163fa1ed6846ff81682c8369ce79a1"
format_stamp: "Formatted at 2026-08-06 07:57:48 on dist-test-slave-g350"
I20260806 07:57:48.765749 3627 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 07:57:48.769757 3642 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:48.771256 3627 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:57:48.771468 3627 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "70163fa1ed6846ff81682c8369ce79a1"
format_stamp: "Formatted at 2026-08-06 07:57:48 on dist-test-slave-g350"
I20260806 07:57:48.771727 3627 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:48.800444 3627 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:48.810094 3627 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:48.810433 3627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:48.832281 3627 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:33853
I20260806 07:57:48.832304 3694 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:33853 every 8 connection(s)
I20260806 07:57:48.833964 3627 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:48.838084 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3627
I20260806 07:57:48.838282 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:48.838522 3695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:48.851756 3695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1: Bootstrap starting.
Aug 06 07:57:48 dist-test-slave-g350 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:48.856148 3695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:48.857687 3695 log.cc:826] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:48.861236 3695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1: No bootstrap required, opened a new log
I20260806 07:57:48.867553 3695 raft_consensus.cc:359] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70163fa1ed6846ff81682c8369ce79a1" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33853 } }
I20260806 07:57:48.867895 3695 raft_consensus.cc:385] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:48.867976 3695 raft_consensus.cc:740] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70163fa1ed6846ff81682c8369ce79a1, State: Initialized, Role: FOLLOWER
I20260806 07:57:48.868592 3695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70163fa1ed6846ff81682c8369ce79a1" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33853 } }
I20260806 07:57:48.868788 3695 raft_consensus.cc:399] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:57:48.868877 3695 raft_consensus.cc:493] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:57:48.868999 3695 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:48.871685 3695 raft_consensus.cc:515] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70163fa1ed6846ff81682c8369ce79a1" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33853 } }
I20260806 07:57:48.872135 3695 leader_election.cc:304] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70163fa1ed6846ff81682c8369ce79a1; no voters:
I20260806 07:57:48.873111 3695 leader_election.cc:290] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:57:48.873258 3700 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:48.874112 3700 raft_consensus.cc:697] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [term 1 LEADER]: Becoming Leader. State: Replica: 70163fa1ed6846ff81682c8369ce79a1, State: Running, Role: LEADER
I20260806 07:57:48.874759 3700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70163fa1ed6846ff81682c8369ce79a1" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33853 } }
I20260806 07:57:48.875695 3695 sys_catalog.cc:565] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:48.877739 3702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70163fa1ed6846ff81682c8369ce79a1. Latest consensus state: current_term: 1 leader_uuid: "70163fa1ed6846ff81682c8369ce79a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70163fa1ed6846ff81682c8369ce79a1" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33853 } } }
I20260806 07:57:48.878043 3702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:48.879186 3701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70163fa1ed6846ff81682c8369ce79a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70163fa1ed6846ff81682c8369ce79a1" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 33853 } } }
I20260806 07:57:48.879426 3701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:48.880730 3705 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:48.886137 3705 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:48.895874 3705 catalog_manager.cc:1383] Generated new cluster ID: 91ab97ee2a4543eca5106f63fd4c1a33
I20260806 07:57:48.895994 3705 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:48.918900 3705 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:48.920120 3705 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:48.938850 3705 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 70163fa1ed6846ff81682c8369ce79a1: Generated new TSK 0
I20260806 07:57:48.940104 3705 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:49.108830 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:33853
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:33619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:49.469389 3724 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:49.469815 3724 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:49.469995 3724 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:49.478660 3724 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:49.478893 3724 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:49.489702 3724 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:33619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:33853
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:49.491401 3724 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:49.492794 3724 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:49.502948 3729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:49.503120 3732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:49.502979 3730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:49.504393 3724 server_base.cc:1061] running on GCE node
I20260806 07:57:49.505024 3724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:49.506055 3724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:49.507299 3724 hybrid_clock.cc:648] HybridClock initialized: now 1786003069507259 us; error 51 us; skew 500 ppm
I20260806 07:57:49.507647 3724 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:49 dist-test-slave-g350 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003069, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:49.510824 3724 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:49.512480 3724 webserver.cc:533] Webserver started at http://127.20.202.193:41061/ using document root <none> and password file <none>
I20260806 07:57:49.513224 3724 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:49.513370 3724 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:49.513792 3724 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:49.516199 3724 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "383356c629f74498bc09d22ee57ac466"
format_stamp: "Formatted at 2026-08-06 07:57:49 on dist-test-slave-g350"
I20260806 07:57:49.516984 3724 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "383356c629f74498bc09d22ee57ac466"
format_stamp: "Formatted at 2026-08-06 07:57:49 on dist-test-slave-g350"
I20260806 07:57:49.522595 3724 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 07:57:49.526093 3739 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:49.527493 3724 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 07:57:49.527681 3724 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "383356c629f74498bc09d22ee57ac466"
format_stamp: "Formatted at 2026-08-06 07:57:49 on dist-test-slave-g350"
I20260806 07:57:49.527892 3724 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:49.538890 3724 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:49.542630 3724 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:49.542936 3724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:49.543884 3724 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:49.545573 3724 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:49.545792 3724 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 07:57:49.545950 3724 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:49.546027 3724 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:49.583168 3724 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38513
I20260806 07:57:49.583227 3851 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38513 every 8 connection(s)
I20260806 07:57:49.584666 3724 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:49.586352 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3724
I20260806 07:57:49.586546 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_3.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:49 dist-test-slave-g350 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003069, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:49.605398 3852 heartbeater.cc:344] Connected to a master server at 127.20.202.254:33853
I20260806 07:57:49.605822 3852 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:49.606635 3852 heartbeater.cc:507] Master 127.20.202.254:33853 requested a full tablet report, sending...
I20260806 07:57:49.608796 3659 ts_manager.cc:194] Registered new tserver with Master: 383356c629f74498bc09d22ee57ac466 (127.20.202.193:38513)
I20260806 07:57:49.610414 3659 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:43113
I20260806 07:57:49.614888 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:57:49 dist-test-slave-g350 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:57:49 dist-test-slave-g350 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:57:49.676854 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3724
Aug 06 07:57:49 dist-test-slave-g350 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:49.693410 3847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:49.780754 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3724
I20260806 07:57:49.799414 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3627
I20260806 07:57:49.800661 3689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:49.909919 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3627
2026-08-06T07:57:49Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/3 (4092 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/4
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:50 dist-test-slave-g350 krb5kdc[3866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:50 dist-test-slave-g350 krb5kdc[3866](info): set up 2 sockets
Aug 06 07:57:50 dist-test-slave-g350 krb5kdc[3866](info): commencing operation
krb5kdc: starting...
W20260806 07:57:52.477363 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.522s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:52 dist-test-slave-g350 krb5kdc[3866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:52.881436 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:41193
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:41193
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:53.252705 3882 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:53.253082 3882 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:53.253250 3882 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:53.262048 3882 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:53.262149 3882 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:53.262264 3882 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:53.262346 3882 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:53.273461 3882 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:41193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:41193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:53.275205 3882 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:53.276626 3882 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:53.285629 3888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:53.285692 3889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:53.286037 3891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:53.287333 3882 server_base.cc:1061] running on GCE node
I20260806 07:57:53.288093 3882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:53.289487 3882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:53.290750 3882 hybrid_clock.cc:648] HybridClock initialized: now 1786003073290729 us; error 71 us; skew 500 ppm
I20260806 07:57:53.291155 3882 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:53 dist-test-slave-g350 krb5kdc[3866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003073, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:53.294344 3882 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:53.295895 3882 webserver.cc:533] Webserver started at http://127.20.202.254:45665/ using document root <none> and password file <none>
I20260806 07:57:53.296636 3882 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:53.296783 3882 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:53.297822 3882 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:53.300354 3882 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "7cd7f6642e204048ad91af5ffa0150f6"
format_stamp: "Formatted at 2026-08-06 07:57:53 on dist-test-slave-g350"
I20260806 07:57:53.301200 3882 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "7cd7f6642e204048ad91af5ffa0150f6"
format_stamp: "Formatted at 2026-08-06 07:57:53 on dist-test-slave-g350"
I20260806 07:57:53.306818 3882 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 07:57:53.310401 3898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:53.311843 3882 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:57:53.312026 3882 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "7cd7f6642e204048ad91af5ffa0150f6"
format_stamp: "Formatted at 2026-08-06 07:57:53 on dist-test-slave-g350"
I20260806 07:57:53.312253 3882 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:53.333876 3882 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:53.337761 3882 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:53.338071 3882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:53.359972 3882 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:41193
I20260806 07:57:53.359994 3949 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:41193 every 8 connection(s)
I20260806 07:57:53.361689 3882 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:53.363129 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3882
I20260806 07:57:53.363324 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:53.367199 3950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 07:57:53 dist-test-slave-g350 krb5kdc[3866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:53.377018 3950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6: Bootstrap starting.
I20260806 07:57:53.380616 3950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6: Neither blocks nor log segments found. Creating new log.
I20260806 07:57:53.381729 3950 log.cc:826] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:53.385120 3950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6: No bootstrap required, opened a new log
I20260806 07:57:53.390628 3950 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd7f6642e204048ad91af5ffa0150f6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 41193 } }
I20260806 07:57:53.390972 3950 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:53.391093 3950 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7f6642e204048ad91af5ffa0150f6, State: Initialized, Role: FOLLOWER
I20260806 07:57:53.391750 3950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd7f6642e204048ad91af5ffa0150f6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 41193 } }
I20260806 07:57:53.391970 3950 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:57:53.392109 3950 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:57:53.392333 3950 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:53.394765 3950 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd7f6642e204048ad91af5ffa0150f6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 41193 } }
I20260806 07:57:53.395222 3950 leader_election.cc:304] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd7f6642e204048ad91af5ffa0150f6; no voters:
I20260806 07:57:53.395829 3950 leader_election.cc:290] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:57:53.395984 3955 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:53.396665 3955 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [term 1 LEADER]: Becoming Leader. State: Replica: 7cd7f6642e204048ad91af5ffa0150f6, State: Running, Role: LEADER
I20260806 07:57:53.397221 3955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd7f6642e204048ad91af5ffa0150f6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 41193 } }
I20260806 07:57:53.398003 3950 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:53.400278 3957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd7f6642e204048ad91af5ffa0150f6. Latest consensus state: current_term: 1 leader_uuid: "7cd7f6642e204048ad91af5ffa0150f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd7f6642e204048ad91af5ffa0150f6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 41193 } } }
I20260806 07:57:53.400584 3957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:53.402467 3956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cd7f6642e204048ad91af5ffa0150f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd7f6642e204048ad91af5ffa0150f6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 41193 } } }
I20260806 07:57:53.402825 3956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6 [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:53.403606 3961 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:53.410281 3961 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:53.419272 3961 catalog_manager.cc:1383] Generated new cluster ID: b15a75656ae04063995b7c1930ea9b02
I20260806 07:57:53.419404 3961 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:53.441110 3961 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:53.442715 3961 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:53.468961 3961 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7cd7f6642e204048ad91af5ffa0150f6: Generated new TSK 0
I20260806 07:57:53.469944 3961 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:53.624998 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:41193
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:53.991359 3979 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:53.991653 3979 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:53.991741 3979 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:54.000326 3979 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:54.000509 3979 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:54.011402 3979 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:41193
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:54.012974 3979 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:54.014310 3979 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:54.024318 3985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:54.024959 3987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:54.025151 3984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:54.025843 3979 server_base.cc:1061] running on GCE node
I20260806 07:57:54.026536 3979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:54.027510 3979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:54.028713 3979 hybrid_clock.cc:648] HybridClock initialized: now 1786003074028676 us; error 27 us; skew 500 ppm
I20260806 07:57:54.029032 3979 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[3866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003074, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:54.032325 3979 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:54.034049 3979 webserver.cc:533] Webserver started at http://127.20.202.193:36163/ using document root <none> and password file <none>
I20260806 07:57:54.034793 3979 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:54.034951 3979 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:54.035367 3979 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:54.037983 3979 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "ed2062d0e41d4feb8cb010ebe3abcf94"
format_stamp: "Formatted at 2026-08-06 07:57:54 on dist-test-slave-g350"
I20260806 07:57:54.038759 3979 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "ed2062d0e41d4feb8cb010ebe3abcf94"
format_stamp: "Formatted at 2026-08-06 07:57:54 on dist-test-slave-g350"
I20260806 07:57:54.044152 3979 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 07:57:54.047672 3994 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:54.049055 3979 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:57:54.049275 3979 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "ed2062d0e41d4feb8cb010ebe3abcf94"
format_stamp: "Formatted at 2026-08-06 07:57:54 on dist-test-slave-g350"
I20260806 07:57:54.049485 3979 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:54.069625 3979 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:54.077117 3979 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:54.077514 3979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:54.078693 3979 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:54.080410 3979 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:54.080525 3979 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:54.080681 3979 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:54.080760 3979 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:54.127104 3979 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:35191
I20260806 07:57:54.127190 4107 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:35191 every 8 connection(s)
I20260806 07:57:54.128589 3979 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:54.131672 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 3979
I20260806 07:57:54.131875 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_4.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[3866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003074, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:54.150611 4108 heartbeater.cc:344] Connected to a master server at 127.20.202.254:41193
I20260806 07:57:54.151055 4108 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:54.151917 4108 heartbeater.cc:507] Master 127.20.202.254:41193 requested a full tablet report, sending...
I20260806 07:57:54.154328 3915 ts_manager.cc:194] Registered new tserver with Master: ed2062d0e41d4feb8cb010ebe3abcf94 (127.20.202.193:35191)
I20260806 07:57:54.156165 3915 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:32801
I20260806 07:57:54.160771 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[3866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[3866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:57:54.244997 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3979
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[3866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:54.262786 4103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:54.351523 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3979
I20260806 07:57:54.374873 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 3882
I20260806 07:57:54.376155 3945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:54.484185 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 3882
2026-08-06T07:57:54Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/4 (4575 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/5
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[4121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[4121](info): set up 2 sockets
Aug 06 07:57:54 dist-test-slave-g350 krb5kdc[4121](info): commencing operation
krb5kdc: starting...
W20260806 07:57:56.614519 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.083s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:57:56 dist-test-slave-g350 krb5kdc[4121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:57:56Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:57.006248 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43901
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:43901
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:57.357125 4137 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:57.357506 4137 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:57.357662 4137 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:57.366102 4137 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:57:57.366210 4137 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:57.366310 4137 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:57:57.366410 4137 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:57:57.377568 4137 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:43901
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:43901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:57.379426 4137 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:57.380980 4137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:57.389111 4143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:57.389184 4142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:57.389568 4145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:57.390214 4137 server_base.cc:1061] running on GCE node
I20260806 07:57:57.390955 4137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:57.392431 4137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:57.393713 4137 hybrid_clock.cc:648] HybridClock initialized: now 1786003077393637 us; error 70 us; skew 500 ppm
I20260806 07:57:57.394124 4137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:57 dist-test-slave-g350 krb5kdc[4121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003077, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:57.397331 4137 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:57:57.399114 4137 webserver.cc:533] Webserver started at http://127.20.202.254:44823/ using document root <none> and password file <none>
I20260806 07:57:57.399852 4137 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:57.399994 4137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:57.400345 4137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:57.402809 4137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "1ddc6f679e824b6f923fdeb5f3879b9e"
format_stamp: "Formatted at 2026-08-06 07:57:57 on dist-test-slave-g350"
I20260806 07:57:57.403612 4137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "1ddc6f679e824b6f923fdeb5f3879b9e"
format_stamp: "Formatted at 2026-08-06 07:57:57 on dist-test-slave-g350"
I20260806 07:57:57.409088 4137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 07:57:57.412773 4152 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:57.414276 4137 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:57:57.414458 4137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "1ddc6f679e824b6f923fdeb5f3879b9e"
format_stamp: "Formatted at 2026-08-06 07:57:57 on dist-test-slave-g350"
I20260806 07:57:57.414696 4137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:57.444718 4137 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:57.449076 4137 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:57.449381 4137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:57.470794 4137 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:43901
I20260806 07:57:57.470839 4204 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:43901 every 8 connection(s)
I20260806 07:57:57.472455 4137 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:57:57.476991 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:57:57.478822 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4137
I20260806 07:57:57.479035 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:57:57.488867 4205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e: Bootstrap starting.
I20260806 07:57:57.493391 4205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e: Neither blocks nor log segments found. Creating new log.
Aug 06 07:57:57 dist-test-slave-g350 krb5kdc[4121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:57.494565 4205 log.cc:826] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e: Log is configured to *not* fsync() on all Append() calls
I20260806 07:57:57.498080 4205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e: No bootstrap required, opened a new log
I20260806 07:57:57.504833 4205 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43901 } }
I20260806 07:57:57.505211 4205 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:57:57.505322 4205 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddc6f679e824b6f923fdeb5f3879b9e, State: Initialized, Role: FOLLOWER
I20260806 07:57:57.505961 4205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43901 } }
I20260806 07:57:57.506189 4205 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:57:57.506337 4205 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:57:57.506529 4205 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:57:57.508670 4205 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43901 } }
I20260806 07:57:57.509171 4205 leader_election.cc:304] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddc6f679e824b6f923fdeb5f3879b9e; no voters:
I20260806 07:57:57.509876 4205 leader_election.cc:290] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:57:57.510048 4210 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:57:57.511755 4210 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [term 1 LEADER]: Becoming Leader. State: Replica: 1ddc6f679e824b6f923fdeb5f3879b9e, State: Running, Role: LEADER
I20260806 07:57:57.512410 4210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43901 } }
I20260806 07:57:57.513065 4205 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:57:57.515833 4212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddc6f679e824b6f923fdeb5f3879b9e. Latest consensus state: current_term: 1 leader_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43901 } } }
I20260806 07:57:57.516202 4212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:57.516920 4211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddc6f679e824b6f923fdeb5f3879b9e" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 43901 } } }
I20260806 07:57:57.517323 4211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e [sys.catalog]: This master's current role is: LEADER
I20260806 07:57:57.518378 4219 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:57:57.524022 4219 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:57:57.533876 4219 catalog_manager.cc:1383] Generated new cluster ID: 26b3b4f0210542c58d4224a7d164594d
I20260806 07:57:57.534071 4219 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:57:57.545805 4219 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:57:57.547583 4219 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:57:57.566123 4219 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1ddc6f679e824b6f923fdeb5f3879b9e: Generated new TSK 0
I20260806 07:57:57.567126 4219 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:57:57.733253 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:43901
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:35725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:57:58.090229 4233 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:57:58.090534 4233 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:57:58.090734 4233 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:57:58.099952 4233 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:57:58.100194 4233 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:57:58.111135 4233 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:35725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:43901
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:57:58.112808 4233 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:57:58.114365 4233 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:57:58.124447 4239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:58.124512 4238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:57:58.124894 4241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:57:58.125694 4233 server_base.cc:1061] running on GCE node
I20260806 07:57:58.126291 4233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:57:58.127321 4233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:57:58.128567 4233 hybrid_clock.cc:648] HybridClock initialized: now 1786003078128529 us; error 55 us; skew 500 ppm
I20260806 07:57:58.128890 4233 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003078, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:57:58.132225 4233 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:57:58.133958 4233 webserver.cc:533] Webserver started at http://127.20.202.193:37991/ using document root <none> and password file <none>
I20260806 07:57:58.134655 4233 fs_manager.cc:362] Metadata directory not provided
I20260806 07:57:58.134792 4233 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:57:58.135187 4233 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:57:58.137707 4233 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "aab46a627dcc4795add3e883597a4122"
format_stamp: "Formatted at 2026-08-06 07:57:58 on dist-test-slave-g350"
I20260806 07:57:58.138540 4233 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "aab46a627dcc4795add3e883597a4122"
format_stamp: "Formatted at 2026-08-06 07:57:58 on dist-test-slave-g350"
I20260806 07:57:58.144075 4233 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 07:57:58.147624 4248 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:58.149072 4233 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:57:58.149264 4233 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "aab46a627dcc4795add3e883597a4122"
format_stamp: "Formatted at 2026-08-06 07:57:58 on dist-test-slave-g350"
I20260806 07:57:58.149484 4233 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:57:58.188027 4233 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:57:58.192427 4233 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:57:58.192756 4233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:57:58.193845 4233 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:57:58.195434 4233 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:57:58.195554 4233 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:58.195715 4233 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:57:58.195792 4233 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:57:58.230443 4233 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:38185
I20260806 07:57:58.230547 4361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:38185 every 8 connection(s)
I20260806 07:57:58.232005 4233 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:57:58.235909 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4233
I20260806 07:57:58.236117 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_5.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003078, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:57:58.254297 4362 heartbeater.cc:344] Connected to a master server at 127.20.202.254:43901
I20260806 07:57:58.254750 4362 heartbeater.cc:461] Registering TS with master...
I20260806 07:57:58.255695 4362 heartbeater.cc:507] Master 127.20.202.254:43901 requested a full tablet report, sending...
I20260806 07:57:58.258153 4169 ts_manager.cc:194] Registered new tserver with Master: aab46a627dcc4795add3e883597a4122 (127.20.202.193:38185)
I20260806 07:57:58.259743 4169 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:48039
I20260806 07:57:58.264678 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:57:58.328125 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4233
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:57:58.346817 4357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:58.436610 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4233
I20260806 07:57:58.456298 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4137
I20260806 07:57:58.457806 4200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:57:58.567397 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4137
2026-08-06T07:57:58Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/5 (4080 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/6
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4375](info): set up 2 sockets
Aug 06 07:57:58 dist-test-slave-g350 krb5kdc[4375](info): commencing operation
krb5kdc: starting...
W20260806 07:58:00.681308 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.070s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:58:00 dist-test-slave-g350 krb5kdc[4375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:58:00Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:01.079280 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:38225
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:38225
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:01.436720 4391 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:01.437085 4391 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:01.437213 4391 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:01.445876 4391 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:01.445979 4391 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:01.446106 4391 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:01.446177 4391 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:58:01.457289 4391 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:38225
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:01.459019 4391 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:01.460438 4391 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:01.469341 4396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:01.469408 4397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:01.469385 4399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:01.470530 4391 server_base.cc:1061] running on GCE node
I20260806 07:58:01.471295 4391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:01.472735 4391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:01.473984 4391 hybrid_clock.cc:648] HybridClock initialized: now 1786003081473921 us; error 58 us; skew 500 ppm
I20260806 07:58:01.474386 4391 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:01 dist-test-slave-g350 krb5kdc[4375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003081, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:01.477582 4391 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:58:01.479341 4391 webserver.cc:533] Webserver started at http://127.20.202.254:43491/ using document root <none> and password file <none>
I20260806 07:58:01.480103 4391 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:01.480240 4391 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:01.480592 4391 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:01.483139 4391 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "668068f9e00d4c1b8030edfeccbca4a6"
format_stamp: "Formatted at 2026-08-06 07:58:01 on dist-test-slave-g350"
I20260806 07:58:01.483908 4391 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "668068f9e00d4c1b8030edfeccbca4a6"
format_stamp: "Formatted at 2026-08-06 07:58:01 on dist-test-slave-g350"
I20260806 07:58:01.489482 4391 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:58:01.492837 4406 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:01.494244 4391 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:01.494426 4391 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "668068f9e00d4c1b8030edfeccbca4a6"
format_stamp: "Formatted at 2026-08-06 07:58:01 on dist-test-slave-g350"
I20260806 07:58:01.494656 4391 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:01.513211 4391 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:01.520936 4391 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:01.521256 4391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:01.543292 4391 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:38225
I20260806 07:58:01.543337 4457 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:38225 every 8 connection(s)
I20260806 07:58:01.544898 4391 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:01.549495 4458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:01.552045 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4391
I20260806 07:58:01.552254 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:01.561241 4458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6: Bootstrap starting.
I20260806 07:58:01.564750 4458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:01.565986 4458 log.cc:826] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6: Log is configured to *not* fsync() on all Append() calls
Aug 06 07:58:01 dist-test-slave-g350 krb5kdc[4375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:01.569109 4458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6: No bootstrap required, opened a new log
I20260806 07:58:01.576200 4458 raft_consensus.cc:359] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668068f9e00d4c1b8030edfeccbca4a6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38225 } }
I20260806 07:58:01.576592 4458 raft_consensus.cc:385] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:01.576727 4458 raft_consensus.cc:740] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668068f9e00d4c1b8030edfeccbca4a6, State: Initialized, Role: FOLLOWER
I20260806 07:58:01.577432 4458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668068f9e00d4c1b8030edfeccbca4a6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38225 } }
I20260806 07:58:01.577700 4458 raft_consensus.cc:399] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:58:01.577847 4458 raft_consensus.cc:493] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:58:01.578035 4458 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:01.580246 4458 raft_consensus.cc:515] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668068f9e00d4c1b8030edfeccbca4a6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38225 } }
I20260806 07:58:01.580781 4458 leader_election.cc:304] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 668068f9e00d4c1b8030edfeccbca4a6; no voters:
I20260806 07:58:01.581516 4458 leader_election.cc:290] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:58:01.581687 4464 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:01.582557 4464 raft_consensus.cc:697] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [term 1 LEADER]: Becoming Leader. State: Replica: 668068f9e00d4c1b8030edfeccbca4a6, State: Running, Role: LEADER
I20260806 07:58:01.583164 4464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668068f9e00d4c1b8030edfeccbca4a6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38225 } }
I20260806 07:58:01.583911 4458 sys_catalog.cc:565] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:01.586158 4466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 668068f9e00d4c1b8030edfeccbca4a6. Latest consensus state: current_term: 1 leader_uuid: "668068f9e00d4c1b8030edfeccbca4a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668068f9e00d4c1b8030edfeccbca4a6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38225 } } }
I20260806 07:58:01.586226 4465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "668068f9e00d4c1b8030edfeccbca4a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "668068f9e00d4c1b8030edfeccbca4a6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 38225 } } }
I20260806 07:58:01.586531 4465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:01.586529 4466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:01.589846 4469 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:01.595501 4469 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:01.604293 4469 catalog_manager.cc:1383] Generated new cluster ID: fe17aeeb0a594e68b572049b24f4c493
I20260806 07:58:01.604427 4469 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:01.618098 4469 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:01.619961 4469 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:01.639950 4469 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 668068f9e00d4c1b8030edfeccbca4a6: Generated new TSK 0
I20260806 07:58:01.640924 4469 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:01.790045 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:38225
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:02.157289 4487 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:02.157680 4487 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:02.157838 4487 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:02.166615 4487 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:02.166860 4487 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:02.177958 4487 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:38225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:02.179687 4487 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:02.181130 4487 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:02.191443 4493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:02.191447 4492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:02.192368 4495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:02.193222 4487 server_base.cc:1061] running on GCE node
I20260806 07:58:02.193980 4487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:02.195209 4487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:02.196416 4487 hybrid_clock.cc:648] HybridClock initialized: now 1786003082196364 us; error 55 us; skew 500 ppm
I20260806 07:58:02.196748 4487 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003082, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:02.200095 4487 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:58:02.201818 4487 webserver.cc:533] Webserver started at http://127.20.202.193:41499/ using document root <none> and password file <none>
I20260806 07:58:02.202527 4487 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:02.202626 4487 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:02.202940 4487 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:02.205399 4487 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "7093938679ee4caeb92eb63d81aa88f6"
format_stamp: "Formatted at 2026-08-06 07:58:02 on dist-test-slave-g350"
I20260806 07:58:02.206205 4487 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "7093938679ee4caeb92eb63d81aa88f6"
format_stamp: "Formatted at 2026-08-06 07:58:02 on dist-test-slave-g350"
I20260806 07:58:02.211654 4487 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:58:02.215107 4502 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:02.216463 4487 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:58:02.216630 4487 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7093938679ee4caeb92eb63d81aa88f6"
format_stamp: "Formatted at 2026-08-06 07:58:02 on dist-test-slave-g350"
I20260806 07:58:02.216818 4487 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:02.246306 4487 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:02.250703 4487 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:02.251025 4487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:02.252130 4487 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:02.253875 4487 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:02.253959 4487 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:02.254086 4487 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:02.254155 4487 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:02.291735 4487 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:32835
I20260806 07:58:02.291785 4614 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:32835 every 8 connection(s)
I20260806 07:58:02.293500 4487 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:58:02.297051 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4487
I20260806 07:58:02.297250 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_6.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003082, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:02.314137 4615 heartbeater.cc:344] Connected to a master server at 127.20.202.254:38225
I20260806 07:58:02.314527 4615 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:02.315371 4615 heartbeater.cc:507] Master 127.20.202.254:38225 requested a full tablet report, sending...
I20260806 07:58:02.317564 4423 ts_manager.cc:194] Registered new tserver with Master: 7093938679ee4caeb92eb63d81aa88f6 (127.20.202.193:32835)
I20260806 07:58:02.319183 4423 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:45407
I20260806 07:58:02.325084 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:58:02.386862 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4487
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:58:02.404446 4610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:02.490218 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4487
I20260806 07:58:02.509529 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4391
I20260806 07:58:02.510845 4453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:02.618933 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4391
2026-08-06T07:58:02Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/6 (4050 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/7
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4629](info): set up 2 sockets
Aug 06 07:58:02 dist-test-slave-g350 krb5kdc[4629](info): commencing operation
krb5kdc: starting...
W20260806 07:58:04.745113 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.082s user 0.007s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:58:04 dist-test-slave-g350 krb5kdc[4629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:58:04Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:05.133369 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45799
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45799
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:05.483845 4645 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:05.484243 4645 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:05.484422 4645 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:05.492719 4645 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:05.492874 4645 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:05.492972 4645 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:05.493070 4645 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:58:05.503960 4645 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45799
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:05.505683 4645 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:05.507117 4645 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:05.515712 4652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:05.515662 4651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:05.516283 4654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:05.516896 4645 server_base.cc:1061] running on GCE node
I20260806 07:58:05.517741 4645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:05.519165 4645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:05.520385 4645 hybrid_clock.cc:648] HybridClock initialized: now 1786003085520327 us; error 52 us; skew 500 ppm
I20260806 07:58:05.520748 4645 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:05 dist-test-slave-g350 krb5kdc[4629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003085, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:05.523947 4645 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:58:05.525583 4645 webserver.cc:533] Webserver started at http://127.20.202.254:36083/ using document root <none> and password file <none>
I20260806 07:58:05.526361 4645 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:05.526500 4645 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:05.526852 4645 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:05.529290 4645 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "ce81002539d748c68c16497e92e78ab6"
format_stamp: "Formatted at 2026-08-06 07:58:05 on dist-test-slave-g350"
I20260806 07:58:05.530153 4645 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "ce81002539d748c68c16497e92e78ab6"
format_stamp: "Formatted at 2026-08-06 07:58:05 on dist-test-slave-g350"
I20260806 07:58:05.535691 4645 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:58:05.539460 4661 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:05.540918 4645 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 07:58:05.541141 4645 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "ce81002539d748c68c16497e92e78ab6"
format_stamp: "Formatted at 2026-08-06 07:58:05 on dist-test-slave-g350"
I20260806 07:58:05.541378 4645 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:05.559690 4645 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:05.564100 4645 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:05.564417 4645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:05.586838 4645 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45799
I20260806 07:58:05.586869 4712 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45799 every 8 connection(s)
I20260806 07:58:05.588632 4645 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:05.593127 4713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:05.595885 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4645
I20260806 07:58:05.596099 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:05.605006 4713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6: Bootstrap starting.
I20260806 07:58:05.609490 4713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6: Neither blocks nor log segments found. Creating new log.
Aug 06 07:58:05 dist-test-slave-g350 krb5kdc[4629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:05.610795 4713 log.cc:826] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:05.613524 4713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6: No bootstrap required, opened a new log
I20260806 07:58:05.619182 4713 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce81002539d748c68c16497e92e78ab6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45799 } }
I20260806 07:58:05.619575 4713 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:05.619714 4713 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce81002539d748c68c16497e92e78ab6, State: Initialized, Role: FOLLOWER
I20260806 07:58:05.620317 4713 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce81002539d748c68c16497e92e78ab6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45799 } }
I20260806 07:58:05.620548 4713 raft_consensus.cc:399] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:58:05.620702 4713 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:58:05.620893 4713 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:05.623224 4713 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce81002539d748c68c16497e92e78ab6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45799 } }
I20260806 07:58:05.623709 4713 leader_election.cc:304] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce81002539d748c68c16497e92e78ab6; no voters:
I20260806 07:58:05.624280 4713 leader_election.cc:290] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:58:05.624419 4718 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:05.625224 4718 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [term 1 LEADER]: Becoming Leader. State: Replica: ce81002539d748c68c16497e92e78ab6, State: Running, Role: LEADER
I20260806 07:58:05.625831 4718 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce81002539d748c68c16497e92e78ab6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45799 } }
I20260806 07:58:05.626492 4713 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:05.629493 4720 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce81002539d748c68c16497e92e78ab6. Latest consensus state: current_term: 1 leader_uuid: "ce81002539d748c68c16497e92e78ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce81002539d748c68c16497e92e78ab6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45799 } } }
I20260806 07:58:05.629886 4720 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:05.631644 4719 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce81002539d748c68c16497e92e78ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce81002539d748c68c16497e92e78ab6" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45799 } } }
I20260806 07:58:05.632102 4719 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:05.632344 4729 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:05.636850 4729 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:05.645804 4729 catalog_manager.cc:1383] Generated new cluster ID: c1ee7995aa8040eb97096fa23b37e3bf
I20260806 07:58:05.645938 4729 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:05.660816 4729 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:05.662336 4729 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:05.674407 4729 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ce81002539d748c68c16497e92e78ab6: Generated new TSK 0
I20260806 07:58:05.675302 4729 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:05.829630 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45799
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:06.190344 4742 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:06.190653 4742 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:06.190822 4742 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:06.199411 4742 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:06.199640 4742 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:06.209930 4742 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45799
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:06.211517 4742 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:06.212872 4742 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:06.223538 4748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:06.223578 4747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:06.225858 4742 server_base.cc:1061] running on GCE node
W20260806 07:58:06.226660 4750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:06.227342 4742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:06.228497 4742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:06.229710 4742 hybrid_clock.cc:648] HybridClock initialized: now 1786003086229648 us; error 41 us; skew 500 ppm
I20260806 07:58:06.230046 4742 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003086, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:06.233449 4742 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:58:06.235194 4742 webserver.cc:533] Webserver started at http://127.20.202.193:45405/ using document root <none> and password file <none>
I20260806 07:58:06.236009 4742 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:06.236152 4742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:06.236517 4742 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:06.239073 4742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "2ebff19ef34c4dc2b4399537eb703e5a"
format_stamp: "Formatted at 2026-08-06 07:58:06 on dist-test-slave-g350"
I20260806 07:58:06.239852 4742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "2ebff19ef34c4dc2b4399537eb703e5a"
format_stamp: "Formatted at 2026-08-06 07:58:06 on dist-test-slave-g350"
I20260806 07:58:06.245477 4742 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:58:06.249032 4757 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:06.250634 4742 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:06.250821 4742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "2ebff19ef34c4dc2b4399537eb703e5a"
format_stamp: "Formatted at 2026-08-06 07:58:06 on dist-test-slave-g350"
I20260806 07:58:06.251050 4742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:06.265731 4742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:06.270004 4742 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:06.270316 4742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:06.271284 4742 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:06.272861 4742 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:06.272976 4742 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:06.273138 4742 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:06.273212 4742 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:06.309863 4742 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:39175
I20260806 07:58:06.309955 4870 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:39175 every 8 connection(s)
I20260806 07:58:06.311383 4742 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:58:06.314675 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4742
I20260806 07:58:06.314877 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_7.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003086, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:06.333063 4871 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45799
I20260806 07:58:06.333431 4871 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:06.334389 4871 heartbeater.cc:507] Master 127.20.202.254:45799 requested a full tablet report, sending...
I20260806 07:58:06.337164 4678 ts_manager.cc:194] Registered new tserver with Master: 2ebff19ef34c4dc2b4399537eb703e5a (127.20.202.193:39175)
I20260806 07:58:06.339746 4678 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:53527
I20260806 07:58:06.344551 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:58:06.417387 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4742
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:58:06.434628 4866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:06.519905 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4742
I20260806 07:58:06.539512 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4645
I20260806 07:58:06.540850 4708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:06.646939 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4645
2026-08-06T07:58:06Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/7 (4028 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/8
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4884](info): set up 2 sockets
Aug 06 07:58:06 dist-test-slave-g350 krb5kdc[4884](info): commencing operation
krb5kdc: starting...
W20260806 07:58:08.796967 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.106s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:58:09 dist-test-slave-g350 krb5kdc[4884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:58:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:09.184239 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34335
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34335
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:09.536079 4900 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:09.536434 4900 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:09.536613 4900 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:09.545051 4900 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:09.545156 4900 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:09.545262 4900 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:09.545352 4900 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:58:09.556238 4900 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34335
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:09.557937 4900 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:09.559382 4900 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:09.567919 4907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:09.568007 4906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:09.569238 4900 server_base.cc:1061] running on GCE node
W20260806 07:58:09.569535 4909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:09.570343 4900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:09.571812 4900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:09.573119 4900 hybrid_clock.cc:648] HybridClock initialized: now 1786003089573028 us; error 73 us; skew 500 ppm
I20260806 07:58:09.573479 4900 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:09 dist-test-slave-g350 krb5kdc[4884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003089, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:09.576623 4900 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:58:09.578485 4900 webserver.cc:533] Webserver started at http://127.20.202.254:43977/ using document root <none> and password file <none>
I20260806 07:58:09.579262 4900 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:09.579406 4900 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:09.579787 4900 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:09.582374 4900 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "c50ca3a72ed1494689285582b5f07918"
format_stamp: "Formatted at 2026-08-06 07:58:09 on dist-test-slave-g350"
I20260806 07:58:09.583258 4900 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "c50ca3a72ed1494689285582b5f07918"
format_stamp: "Formatted at 2026-08-06 07:58:09 on dist-test-slave-g350"
I20260806 07:58:09.589072 4900 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:58:09.592732 4916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:09.594198 4900 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:09.594390 4900 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "c50ca3a72ed1494689285582b5f07918"
format_stamp: "Formatted at 2026-08-06 07:58:09 on dist-test-slave-g350"
I20260806 07:58:09.594627 4900 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:09.621659 4900 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:09.625450 4900 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:09.625788 4900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:09.647087 4900 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34335
I20260806 07:58:09.647110 4968 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34335 every 8 connection(s)
I20260806 07:58:09.648718 4900 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:09.653076 4969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:09.655702 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4900
I20260806 07:58:09.655920 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:09.665333 4969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918: Bootstrap starting.
I20260806 07:58:09.669085 4969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:09.670382 4969 log.cc:826] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918: Log is configured to *not* fsync() on all Append() calls
Aug 06 07:58:09 dist-test-slave-g350 krb5kdc[4884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:09.673223 4969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918: No bootstrap required, opened a new log
I20260806 07:58:09.680503 4969 raft_consensus.cc:359] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ca3a72ed1494689285582b5f07918" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34335 } }
I20260806 07:58:09.680922 4969 raft_consensus.cc:385] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:09.681041 4969 raft_consensus.cc:740] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50ca3a72ed1494689285582b5f07918, State: Initialized, Role: FOLLOWER
I20260806 07:58:09.681726 4969 consensus_queue.cc:260] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ca3a72ed1494689285582b5f07918" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34335 } }
I20260806 07:58:09.681993 4969 raft_consensus.cc:399] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:58:09.682185 4969 raft_consensus.cc:493] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:58:09.682335 4969 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:09.684505 4969 raft_consensus.cc:515] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ca3a72ed1494689285582b5f07918" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34335 } }
I20260806 07:58:09.685070 4969 leader_election.cc:304] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c50ca3a72ed1494689285582b5f07918; no voters:
I20260806 07:58:09.685778 4969 leader_election.cc:290] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:58:09.685921 4974 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:09.686656 4974 raft_consensus.cc:697] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [term 1 LEADER]: Becoming Leader. State: Replica: c50ca3a72ed1494689285582b5f07918, State: Running, Role: LEADER
I20260806 07:58:09.687176 4974 consensus_queue.cc:237] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ca3a72ed1494689285582b5f07918" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34335 } }
I20260806 07:58:09.687878 4969 sys_catalog.cc:565] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:09.690492 4976 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c50ca3a72ed1494689285582b5f07918. Latest consensus state: current_term: 1 leader_uuid: "c50ca3a72ed1494689285582b5f07918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ca3a72ed1494689285582b5f07918" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34335 } } }
I20260806 07:58:09.690886 4976 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:09.690855 4975 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c50ca3a72ed1494689285582b5f07918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ca3a72ed1494689285582b5f07918" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34335 } } }
I20260806 07:58:09.694027 4975 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:09.698531 4984 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:09.703930 4984 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:09.714059 4984 catalog_manager.cc:1383] Generated new cluster ID: f44cdd674494492287c0ab037b14812c
I20260806 07:58:09.714193 4984 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:09.734295 4984 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:09.735664 4984 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:09.753039 4984 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c50ca3a72ed1494689285582b5f07918: Generated new TSK 0
I20260806 07:58:09.754410 4984 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:09.921540 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34335
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:10.289388 4998 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:10.289793 4998 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:10.289952 4998 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:10.298846 4998 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:10.299091 4998 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:10.310184 4998 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34335
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:10.311924 4998 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:10.313390 4998 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:10.323573 5004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:10.323827 5003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:10.324810 5006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:10.325132 4998 server_base.cc:1061] running on GCE node
I20260806 07:58:10.325764 4998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:10.326660 4998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:10.327878 4998 hybrid_clock.cc:648] HybridClock initialized: now 1786003090327834 us; error 32 us; skew 500 ppm
I20260806 07:58:10.328203 4998 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[4884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003090, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:10.331447 4998 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:58:10.333148 4998 webserver.cc:533] Webserver started at http://127.20.202.193:46225/ using document root <none> and password file <none>
I20260806 07:58:10.333982 4998 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:10.334086 4998 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:10.334431 4998 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:10.336891 4998 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "0699a0abc6a84a458be4127d778781c5"
format_stamp: "Formatted at 2026-08-06 07:58:10 on dist-test-slave-g350"
I20260806 07:58:10.337800 4998 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "0699a0abc6a84a458be4127d778781c5"
format_stamp: "Formatted at 2026-08-06 07:58:10 on dist-test-slave-g350"
I20260806 07:58:10.343292 4998 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 07:58:10.346860 5013 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:10.348268 4998 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 07:58:10.348451 4998 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "0699a0abc6a84a458be4127d778781c5"
format_stamp: "Formatted at 2026-08-06 07:58:10 on dist-test-slave-g350"
I20260806 07:58:10.348663 4998 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:10.366645 4998 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:10.370994 4998 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:10.371309 4998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:10.372325 4998 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:10.373974 4998 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:10.374092 4998 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:10.374246 4998 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:10.374320 4998 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:10.410717 4998 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33069
I20260806 07:58:10.410774 5127 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33069 every 8 connection(s)
I20260806 07:58:10.412173 4998 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:58:10.419327 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 4998
I20260806 07:58:10.419535 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_8.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[4884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003090, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:10.433776 5128 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34335
I20260806 07:58:10.434249 5128 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:10.435364 5128 heartbeater.cc:507] Master 127.20.202.254:34335 requested a full tablet report, sending...
I20260806 07:58:10.437826 4933 ts_manager.cc:194] Registered new tserver with Master: 0699a0abc6a84a458be4127d778781c5 (127.20.202.193:33069)
I20260806 07:58:10.439437 4933 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:43879
I20260806 07:58:10.448022 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[4884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[4884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:58:10.511196 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4998
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[4884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:58:10.528857 5123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:10.616194 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4998
I20260806 07:58:10.635593 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 4900
I20260806 07:58:10.636830 4964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:10.743443 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 4900
2026-08-06T07:58:10Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/8 (4096 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/9
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[5141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[5141](info): set up 2 sockets
Aug 06 07:58:10 dist-test-slave-g350 krb5kdc[5141](info): commencing operation
krb5kdc: starting...
W20260806 07:58:12.857364 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.070s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:58:13 dist-test-slave-g350 krb5kdc[5141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:58:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:13.241298 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42529
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:42529
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:13.591244 5157 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:13.591655 5157 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:13.591814 5157 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:13.600260 5157 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:13.600421 5157 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:13.600544 5157 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:13.600637 5157 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:58:13.611779 5157 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:42529
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:42529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:13.613493 5157 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:13.614969 5157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:13.623234 5163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:13.623507 5162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:13.623980 5165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:13.625108 5157 server_base.cc:1061] running on GCE node
I20260806 07:58:13.626046 5157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:13.627408 5157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:13.628659 5157 hybrid_clock.cc:648] HybridClock initialized: now 1786003093628592 us; error 53 us; skew 500 ppm
I20260806 07:58:13.629015 5157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:13 dist-test-slave-g350 krb5kdc[5141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003093, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:13.632300 5157 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:58:13.634007 5157 webserver.cc:533] Webserver started at http://127.20.202.254:44877/ using document root <none> and password file <none>
I20260806 07:58:13.634779 5157 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:13.634930 5157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:13.635344 5157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:13.637985 5157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "6f08b2f4a90549da9c7b565502653274"
format_stamp: "Formatted at 2026-08-06 07:58:13 on dist-test-slave-g350"
I20260806 07:58:13.638832 5157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "6f08b2f4a90549da9c7b565502653274"
format_stamp: "Formatted at 2026-08-06 07:58:13 on dist-test-slave-g350"
I20260806 07:58:13.644703 5157 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:58:13.648053 5172 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:13.649341 5157 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:13.649525 5157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "6f08b2f4a90549da9c7b565502653274"
format_stamp: "Formatted at 2026-08-06 07:58:13 on dist-test-slave-g350"
I20260806 07:58:13.649811 5157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:13.664116 5157 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:13.673454 5157 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:13.673813 5157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:13.695428 5157 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:42529
I20260806 07:58:13.695463 5224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:42529 every 8 connection(s)
I20260806 07:58:13.697073 5157 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:13.701504 5226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:13.703415 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5157
I20260806 07:58:13.703625 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:13.713687 5226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274: Bootstrap starting.
I20260806 07:58:13.717648 5226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274: Neither blocks nor log segments found. Creating new log.
Aug 06 07:58:13 dist-test-slave-g350 krb5kdc[5141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:13.718894 5226 log.cc:826] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:13.721997 5226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274: No bootstrap required, opened a new log
I20260806 07:58:13.728597 5226 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f08b2f4a90549da9c7b565502653274" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42529 } }
I20260806 07:58:13.729005 5226 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:13.729148 5226 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f08b2f4a90549da9c7b565502653274, State: Initialized, Role: FOLLOWER
I20260806 07:58:13.729954 5226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f08b2f4a90549da9c7b565502653274" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42529 } }
I20260806 07:58:13.730209 5226 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:58:13.730372 5226 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:58:13.730561 5226 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:13.733018 5226 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f08b2f4a90549da9c7b565502653274" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42529 } }
I20260806 07:58:13.733579 5226 leader_election.cc:304] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f08b2f4a90549da9c7b565502653274; no voters:
I20260806 07:58:13.734292 5226 leader_election.cc:290] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:58:13.734396 5231 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:13.735251 5231 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [term 1 LEADER]: Becoming Leader. State: Replica: 6f08b2f4a90549da9c7b565502653274, State: Running, Role: LEADER
I20260806 07:58:13.735828 5231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f08b2f4a90549da9c7b565502653274" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42529 } }
I20260806 07:58:13.736624 5226 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:13.739209 5233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f08b2f4a90549da9c7b565502653274. Latest consensus state: current_term: 1 leader_uuid: "6f08b2f4a90549da9c7b565502653274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f08b2f4a90549da9c7b565502653274" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42529 } } }
I20260806 07:58:13.739619 5233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:13.741897 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f08b2f4a90549da9c7b565502653274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f08b2f4a90549da9c7b565502653274" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 42529 } } }
I20260806 07:58:13.742172 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:13.742780 5238 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:13.748006 5238 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:13.757513 5238 catalog_manager.cc:1383] Generated new cluster ID: f6108467a40144c2b03ee9c8e695f4c9
I20260806 07:58:13.757707 5238 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:13.778568 5238 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:13.780049 5238 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:13.794457 5238 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6f08b2f4a90549da9c7b565502653274: Generated new TSK 0
I20260806 07:58:13.795495 5238 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:13.957209 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:42529
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:14.316682 5254 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:14.317019 5254 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:14.317193 5254 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:14.325659 5254 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:14.325917 5254 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:14.337163 5254 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:42529
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:14.339051 5254 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:14.340487 5254 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:14.350831 5259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:14.350857 5260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:14.351282 5262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:14.352061 5254 server_base.cc:1061] running on GCE node
I20260806 07:58:14.352630 5254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:14.353554 5254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:14.354830 5254 hybrid_clock.cc:648] HybridClock initialized: now 1786003094354758 us; error 56 us; skew 500 ppm
I20260806 07:58:14.355160 5254 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003094, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:14.358415 5254 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:58:14.360108 5254 webserver.cc:533] Webserver started at http://127.20.202.193:44913/ using document root <none> and password file <none>
I20260806 07:58:14.360872 5254 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:14.360997 5254 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:14.361389 5254 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:14.363900 5254 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "39f407bd946e4454b374496ea013bd9d"
format_stamp: "Formatted at 2026-08-06 07:58:14 on dist-test-slave-g350"
I20260806 07:58:14.364667 5254 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "39f407bd946e4454b374496ea013bd9d"
format_stamp: "Formatted at 2026-08-06 07:58:14 on dist-test-slave-g350"
I20260806 07:58:14.370183 5254 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 07:58:14.373668 5269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:14.374984 5254 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:58:14.375160 5254 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "39f407bd946e4454b374496ea013bd9d"
format_stamp: "Formatted at 2026-08-06 07:58:14 on dist-test-slave-g350"
I20260806 07:58:14.375358 5254 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:14.393910 5254 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:14.398156 5254 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:14.398475 5254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:14.399515 5254 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:14.401196 5254 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:14.401276 5254 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:14.401436 5254 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:14.401505 5254 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:14.437991 5254 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:40963
I20260806 07:58:14.438067 5382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:40963 every 8 connection(s)
I20260806 07:58:14.439471 5254 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:58:14.442986 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5254
I20260806 07:58:14.443184 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_9.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003094, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:14.460225 5383 heartbeater.cc:344] Connected to a master server at 127.20.202.254:42529
I20260806 07:58:14.460547 5383 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:14.461329 5383 heartbeater.cc:507] Master 127.20.202.254:42529 requested a full tablet report, sending...
I20260806 07:58:14.463533 5189 ts_manager.cc:194] Registered new tserver with Master: 39f407bd946e4454b374496ea013bd9d (127.20.202.193:40963)
I20260806 07:58:14.465085 5189 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:45135
I20260806 07:58:14.471333 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:58:14.539173 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5254
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:58:14.555742 5378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:14.641491 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5254
I20260806 07:58:14.660436 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5157
I20260806 07:58:14.661860 5220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:14.768603 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5157
2026-08-06T07:58:14Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/9 (4025 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/10
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5396](info): set up 2 sockets
Aug 06 07:58:14 dist-test-slave-g350 krb5kdc[5396](info): commencing operation
krb5kdc: starting...
W20260806 07:58:16.897205 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.084s user 0.004s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:58:17 dist-test-slave-g350 krb5kdc[5396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:58:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:17.293007 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34735
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:34735
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:17.647289 5412 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:17.647647 5412 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:17.647815 5412 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:17.656435 5412 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:17.656548 5412 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:17.656641 5412 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:17.656721 5412 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:58:17.667858 5412 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:34735
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:34735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:17.669642 5412 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:17.671144 5412 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:17.679596 5418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:17.680147 5417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:17.680751 5420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:17.681157 5412 server_base.cc:1061] running on GCE node
I20260806 07:58:17.681942 5412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:17.683329 5412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:17.684585 5412 hybrid_clock.cc:648] HybridClock initialized: now 1786003097684507 us; error 66 us; skew 500 ppm
I20260806 07:58:17.684929 5412 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:17 dist-test-slave-g350 krb5kdc[5396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003097, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:17.688223 5412 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:58:17.689965 5412 webserver.cc:533] Webserver started at http://127.20.202.254:41335/ using document root <none> and password file <none>
I20260806 07:58:17.690749 5412 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:17.690893 5412 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:17.691267 5412 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:17.693849 5412 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "eff640dda04d42fc96e7841361134154"
format_stamp: "Formatted at 2026-08-06 07:58:17 on dist-test-slave-g350"
I20260806 07:58:17.694682 5412 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "eff640dda04d42fc96e7841361134154"
format_stamp: "Formatted at 2026-08-06 07:58:17 on dist-test-slave-g350"
I20260806 07:58:17.700317 5412 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:58:17.704173 5427 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:17.705811 5412 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:17.706003 5412 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "eff640dda04d42fc96e7841361134154"
format_stamp: "Formatted at 2026-08-06 07:58:17 on dist-test-slave-g350"
I20260806 07:58:17.706244 5412 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:17.728654 5412 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:17.732918 5412 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:17.733255 5412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:17.755169 5412 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:34735
I20260806 07:58:17.755215 5479 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:34735 every 8 connection(s)
I20260806 07:58:17.756771 5412 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:17.761497 5480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:17.765906 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5412
I20260806 07:58:17.766111 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:17.772373 5480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154: Bootstrap starting.
I20260806 07:58:17.775698 5480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:17.776786 5480 log.cc:826] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154: Log is configured to *not* fsync() on all Append() calls
Aug 06 07:58:17 dist-test-slave-g350 krb5kdc[5396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:17.779887 5480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154: No bootstrap required, opened a new log
I20260806 07:58:17.786809 5480 raft_consensus.cc:359] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff640dda04d42fc96e7841361134154" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34735 } }
I20260806 07:58:17.787379 5480 raft_consensus.cc:385] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:17.787516 5480 raft_consensus.cc:740] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff640dda04d42fc96e7841361134154, State: Initialized, Role: FOLLOWER
I20260806 07:58:17.788244 5480 consensus_queue.cc:260] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff640dda04d42fc96e7841361134154" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34735 } }
I20260806 07:58:17.788487 5480 raft_consensus.cc:399] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:58:17.788695 5480 raft_consensus.cc:493] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:58:17.788909 5480 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:17.791617 5480 raft_consensus.cc:515] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff640dda04d42fc96e7841361134154" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34735 } }
I20260806 07:58:17.792076 5480 leader_election.cc:304] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eff640dda04d42fc96e7841361134154; no voters:
I20260806 07:58:17.792654 5480 leader_election.cc:290] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:58:17.793197 5486 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:17.794399 5486 raft_consensus.cc:697] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [term 1 LEADER]: Becoming Leader. State: Replica: eff640dda04d42fc96e7841361134154, State: Running, Role: LEADER
I20260806 07:58:17.794895 5486 consensus_queue.cc:237] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff640dda04d42fc96e7841361134154" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34735 } }
I20260806 07:58:17.795372 5480 sys_catalog.cc:565] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:17.797708 5488 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eff640dda04d42fc96e7841361134154. Latest consensus state: current_term: 1 leader_uuid: "eff640dda04d42fc96e7841361134154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff640dda04d42fc96e7841361134154" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34735 } } }
I20260806 07:58:17.798099 5487 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eff640dda04d42fc96e7841361134154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff640dda04d42fc96e7841361134154" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 34735 } } }
I20260806 07:58:17.798523 5487 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:17.798100 5488 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:17.800608 5494 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:17.806488 5494 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:17.817098 5494 catalog_manager.cc:1383] Generated new cluster ID: 22f7ddb84f4f4a5e9d8a09fd6c08e0a7
I20260806 07:58:17.817248 5494 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:17.827275 5494 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:17.829223 5494 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:17.847607 5494 catalog_manager.cc:6070] T 00000000000000000000000000000000 P eff640dda04d42fc96e7841361134154: Generated new TSK 0
I20260806 07:58:17.848800 5494 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:17.997294 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:34735
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:45865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:18.359172 5509 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:18.359490 5509 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:18.359584 5509 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:18.368189 5509 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:18.368391 5509 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:18.379227 5509 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:45865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:34735
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:18.380940 5509 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:18.382325 5509 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:18.392299 5516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:18.392300 5515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:18.393062 5518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:18.393770 5509 server_base.cc:1061] running on GCE node
I20260806 07:58:18.394706 5509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:18.395920 5509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:18.397238 5509 hybrid_clock.cc:648] HybridClock initialized: now 1786003098397163 us; error 67 us; skew 500 ppm
I20260806 07:58:18.397697 5509 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003098, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:18.401548 5509 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:58:18.403564 5509 webserver.cc:533] Webserver started at http://127.20.202.193:38659/ using document root <none> and password file <none>
I20260806 07:58:18.404435 5509 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:18.404575 5509 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:18.404979 5509 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:18.407872 5509 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "33a6f5a3667b4405a1257404927608b4"
format_stamp: "Formatted at 2026-08-06 07:58:18 on dist-test-slave-g350"
I20260806 07:58:18.408772 5509 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "33a6f5a3667b4405a1257404927608b4"
format_stamp: "Formatted at 2026-08-06 07:58:18 on dist-test-slave-g350"
I20260806 07:58:18.415299 5509 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 07:58:18.419303 5525 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:18.420902 5509 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:58:18.421224 5509 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "33a6f5a3667b4405a1257404927608b4"
format_stamp: "Formatted at 2026-08-06 07:58:18 on dist-test-slave-g350"
I20260806 07:58:18.421423 5509 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:18.455938 5509 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:18.460289 5509 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:18.460640 5509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:18.461761 5509 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:18.463536 5509 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:18.463676 5509 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:18.463826 5509 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:18.463901 5509 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:18.499451 5509 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:37805
I20260806 07:58:18.499511 5639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:37805 every 8 connection(s)
I20260806 07:58:18.500957 5509 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003098, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:18.525442 5640 heartbeater.cc:344] Connected to a master server at 127.20.202.254:34735
I20260806 07:58:18.525933 5640 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:18.527192 5640 heartbeater.cc:507] Master 127.20.202.254:34735 requested a full tablet report, sending...
I20260806 07:58:18.528082 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5509
I20260806 07:58:18.528362 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_10.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:58:18.530153 5444 ts_manager.cc:194] Registered new tserver with Master: 33a6f5a3667b4405a1257404927608b4 (127.20.202.193:37805)
I20260806 07:58:18.531929 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260806 07:58:18.532207 5444 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:33391
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:58:18.600024 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5509
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:58:18.617280 5635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:18.704555 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5509
I20260806 07:58:18.723822 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5412
I20260806 07:58:18.725049 5475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:18.833433 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5412
2026-08-06T07:58:18Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/10 (4061 ms)
[ RUN ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/11
Loading random data
Initializing database '/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5653](info): set up 2 sockets
Aug 06 07:58:18 dist-test-slave-g350 krb5kdc[5653](info): commencing operation
krb5kdc: starting...
W20260806 07:58:20.945258 21291 mini_kdc.cc:121] Time spent starting KDC: real 2.071s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/test-user.keytab.
Aug 06 07:58:21 dist-test-slave-g350 krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T07:58:21Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:21.336745 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45939
--webserver_interface=127.20.202.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:45939
--enable_rest_api=true with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:21.697434 5669 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:21.697911 5669 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:21.698073 5669 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:21.706372 5669 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:21.706528 5669 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:21.706632 5669 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:21.706719 5669 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260806 07:58:21.717564 5669 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal
--enable_rest_api=true
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:45939
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--webserver_require_spnego=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:21.719259 5669 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:21.720665 5669 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:21.728883 5675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:21.729044 5674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:21.729625 5677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:21.730480 5669 server_base.cc:1061] running on GCE node
I20260806 07:58:21.731160 5669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:21.732544 5669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:21.733796 5669 hybrid_clock.cc:648] HybridClock initialized: now 1786003101733745 us; error 63 us; skew 500 ppm
I20260806 07:58:21.734149 5669 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:21 dist-test-slave-g350 krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003101, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:21.737382 5669 init.cc:377] Logged in from keytab as kudu/127.20.202.254@KRBTEST.COM (short username kudu)
I20260806 07:58:21.739087 5669 webserver.cc:533] Webserver started at http://127.20.202.254:39639/ using document root <none> and password file <none>
I20260806 07:58:21.739780 5669 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:21.739873 5669 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:21.740181 5669 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:21.742650 5669 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "b59b5745e21042bfb8de7398877df35f"
format_stamp: "Formatted at 2026-08-06 07:58:21 on dist-test-slave-g350"
I20260806 07:58:21.743394 5669 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "b59b5745e21042bfb8de7398877df35f"
format_stamp: "Formatted at 2026-08-06 07:58:21 on dist-test-slave-g350"
I20260806 07:58:21.748826 5669 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 07:58:21.752331 5684 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:21.753880 5669 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:21.754032 5669 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "b59b5745e21042bfb8de7398877df35f"
format_stamp: "Formatted at 2026-08-06 07:58:21 on dist-test-slave-g350"
I20260806 07:58:21.754259 5669 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:21.773869 5669 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:21.777881 5669 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:21.778121 5669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:21.802752 5669 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:45939
I20260806 07:58:21.802790 5736 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:45939 every 8 connection(s)
I20260806 07:58:21.804329 5669 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:21.808816 5737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:21.810194 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5669
I20260806 07:58:21.810400 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:21.820021 5737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f: Bootstrap starting.
I20260806 07:58:21.823637 5737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f: Neither blocks nor log segments found. Creating new log.
Aug 06 07:58:21 dist-test-slave-g350 krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:21.825070 5737 log.cc:826] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:21.828262 5737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f: No bootstrap required, opened a new log
I20260806 07:58:21.835248 5737 raft_consensus.cc:359] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b5745e21042bfb8de7398877df35f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45939 } }
I20260806 07:58:21.835597 5737 raft_consensus.cc:385] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:21.835739 5737 raft_consensus.cc:740] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b59b5745e21042bfb8de7398877df35f, State: Initialized, Role: FOLLOWER
I20260806 07:58:21.836409 5737 consensus_queue.cc:260] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b5745e21042bfb8de7398877df35f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45939 } }
I20260806 07:58:21.836627 5737 raft_consensus.cc:399] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 07:58:21.836755 5737 raft_consensus.cc:493] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 07:58:21.836925 5737 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:21.839200 5737 raft_consensus.cc:515] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b5745e21042bfb8de7398877df35f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45939 } }
I20260806 07:58:21.839758 5737 leader_election.cc:304] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b59b5745e21042bfb8de7398877df35f; no voters:
I20260806 07:58:21.840448 5737 leader_election.cc:290] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 07:58:21.840580 5742 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:21.841310 5742 raft_consensus.cc:697] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [term 1 LEADER]: Becoming Leader. State: Replica: b59b5745e21042bfb8de7398877df35f, State: Running, Role: LEADER
I20260806 07:58:21.841944 5742 consensus_queue.cc:237] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b5745e21042bfb8de7398877df35f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45939 } }
I20260806 07:58:21.842679 5737 sys_catalog.cc:565] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:21.845003 5743 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b59b5745e21042bfb8de7398877df35f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b5745e21042bfb8de7398877df35f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45939 } } }
I20260806 07:58:21.845103 5744 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b59b5745e21042bfb8de7398877df35f. Latest consensus state: current_term: 1 leader_uuid: "b59b5745e21042bfb8de7398877df35f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b5745e21042bfb8de7398877df35f" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 45939 } } }
I20260806 07:58:21.845449 5743 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:21.845510 5744 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:21.846485 5749 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:21.854650 5749 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:21.868359 5749 catalog_manager.cc:1383] Generated new cluster ID: c2a74c9990e341219a32792ffcacd860
I20260806 07:58:21.868508 5749 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:21.877699 5749 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:21.879459 5749 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:21.893493 5749 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b59b5745e21042bfb8de7398877df35f: Generated new TSK 0
I20260806 07:58:21.894639 5749 catalog_manager.cc:1550] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal kudu/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
WARNING: no policy specified for HTTP/127.20.202.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.202.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
Entry for principal HTTP/127.20.202.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab.
I20260806 07:58:22.045099 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:45939
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--webserver_require_spnego=true
--builtin_ntp_servers=127.20.202.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {KRB5CCNAME=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 07:58:22.400604 5765 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:22.401031 5765 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:22.401194 5765 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:22.410000 5765 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:22.410254 5765 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:22.421319 5765 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
--keytab_file=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/krb5kdc/kudu.keytab
--principal=kudu/127.20.202.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--webserver_require_spnego=true
--tserver_master_addrs=127.20.202.254:45939
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:22.422991 5765 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:22.424451 5765 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:22.434973 5771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:22.435061 5770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:22.435303 5773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:22.435870 5765 server_base.cc:1061] running on GCE node
I20260806 07:58:22.436564 5765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:22.437582 5765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:22.438848 5765 hybrid_clock.cc:648] HybridClock initialized: now 1786003102438794 us; error 64 us; skew 500 ppm
I20260806 07:58:22.439203 5765 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 07:58:22 dist-test-slave-g350 krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003102, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 07:58:22.442473 5765 init.cc:377] Logged in from keytab as kudu/127.20.202.193@KRBTEST.COM (short username kudu)
I20260806 07:58:22.444341 5765 webserver.cc:533] Webserver started at http://127.20.202.193:38697/ using document root <none> and password file <none>
I20260806 07:58:22.445116 5765 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:22.445259 5765 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:22.445719 5765 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:22.448323 5765 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "c5f1b603e563459b83b5a97b98c5d301"
format_stamp: "Formatted at 2026-08-06 07:58:22 on dist-test-slave-g350"
I20260806 07:58:22.449117 5765 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "c5f1b603e563459b83b5a97b98c5d301"
format_stamp: "Formatted at 2026-08-06 07:58:22 on dist-test-slave-g350"
I20260806 07:58:22.454818 5765 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 07:58:22.458545 5781 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:22.460038 5765 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:58:22.460253 5765 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "c5f1b603e563459b83b5a97b98c5d301"
format_stamp: "Formatted at 2026-08-06 07:58:22 on dist-test-slave-g350"
I20260806 07:58:22.460449 5765 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:22.480500 5765 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:22.484459 5765 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:22.484772 5765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:22.485867 5765 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:22.487454 5765 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:22.487571 5765 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:22.487726 5765 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:22.487799 5765 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:22.523888 5765 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:33949
I20260806 07:58:22.523985 5893 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:33949 every 8 connection(s)
I20260806 07:58:22.525410 5765 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:58:22.528622 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5765
I20260806 07:58:22.528815 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl_11.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
Aug 06 07:58:22 dist-test-slave-g350 krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003102, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.202.193@KRBTEST.COM for kudu/127.20.202.254@KRBTEST.COM
I20260806 07:58:22.547675 5894 heartbeater.cc:344] Connected to a master server at 127.20.202.254:45939
I20260806 07:58:22.548076 5894 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:22.548903 5894 heartbeater.cc:507] Master 127.20.202.254:45939 requested a full tablet report, sending...
I20260806 07:58:22.551218 5701 ts_manager.cc:194] Registered new tserver with Master: c5f1b603e563459b83b5a97b98c5d301 (127.20.202.193:33949)
I20260806 07:58:22.552852 5701 master_service.cc:502] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.202.193@KRBTEST.COM'} at 127.20.202.193:59067
I20260806 07:58:22.557703 21291 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
Aug 06 07:58:22 dist-test-slave-g350 krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 07:58:22 dist-test-slave-g350 krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for HTTP/127.20.202.254@KRBTEST.COM
I20260806 07:58:22.623337 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5765
Aug 06 07:58:22 dist-test-slave-g350 krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786003102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.202.193@KRBTEST.COM
I20260806 07:58:22.641304 5889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:22.731055 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5765
I20260806 07:58:22.751112 21291 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskUU61cc/build/asan/bin/kudu pid 5669
I20260806 07:58:22.752278 5732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 07:58:22.858662 21291 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskUU61cc/build/asan/bin/kudu with pid 5669
2026-08-06T07:58:22Z chronyd exiting
[ OK ] ExternalMiniClusterTest.TestRestApiSpnegoConnectionThroughCurl/11 (4027 ms)
[----------] 37 tests from ExternalMiniClusterTest (630940 ms total)
[----------] 1 test from Kudu3762Test
[ RUN ] Kudu3762Test.AddMaster
2026-08-06T07:58:22Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-06T07:58:22Z Disabled control of system clock
I20260806 07:58:22.953343 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36591
--webserver_interface=127.20.202.254
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36591,127.20.202.253:37991
--flush_threshold_mb=0
--flush_threshold_secs=1
--log_cache_size_limit_mb=1
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_segment_size_mb=1
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--unlock_experimental_flags=true with env {}
W20260806 07:58:23.309712 5908 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:23.310056 5908 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:23.310261 5908 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:23.319016 5908 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260806 07:58:23.319120 5908 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260806 07:58:23.319263 5908 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:23.319341 5908 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:23.319459 5908 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:23.319543 5908 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:58:23.319686 5908 flags.cc:432] Enabled experimental flag: --flush_threshold_mb=0
W20260806 07:58:23.319751 5908 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=1
I20260806 07:58:23.330453 5908 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36591,127.20.202.253:37991
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.254:36591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.202.254
--webserver_port=0
--flush_threshold_mb=0
--flush_threshold_secs=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
W20260806 07:58:23.330842 5908 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.202.254:36591,127.20.202.253:37991'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 07:58:23.332345 5908 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:23.334307 5908 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:23.342773 5915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:23.342715 5913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:23.343580 5917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:23.344450 5908 server_base.cc:1061] running on GCE node
I20260806 07:58:23.345278 5908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:23.346762 5908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:23.348047 5908 hybrid_clock.cc:648] HybridClock initialized: now 1786003103347985 us; error 39 us; skew 500 ppm
I20260806 07:58:23.348443 5908 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:58:23.350829 5908 webserver.cc:533] Webserver started at http://127.20.202.254:38119/ using document root <none> and password file <none>
I20260806 07:58:23.351615 5908 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:23.351764 5908 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:23.352116 5908 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:23.354611 5908 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data/instance:
uuid: "a3d2d029e02f42bdb166bf54079a7128"
format_stamp: "Formatted at 2026-08-06 07:58:23 on dist-test-slave-g350"
I20260806 07:58:23.355424 5908 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal/instance:
uuid: "a3d2d029e02f42bdb166bf54079a7128"
format_stamp: "Formatted at 2026-08-06 07:58:23 on dist-test-slave-g350"
I20260806 07:58:23.360749 5908 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:58:23.364223 5923 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:23.365511 5908 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:23.365728 5908 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal
uuid: "a3d2d029e02f42bdb166bf54079a7128"
format_stamp: "Formatted at 2026-08-06 07:58:23 on dist-test-slave-g350"
I20260806 07:58:23.365922 5908 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:23.389835 5908 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:23.390723 5908 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:23.391039 5908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:23.414341 5908 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.254:36591
I20260806 07:58:23.414364 5977 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.254:36591 every 8 connection(s)
I20260806 07:58:23.415998 5908 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/data/info.pb
I20260806 07:58:23.420535 5978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:23.426467 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5908
I20260806 07:58:23.426730 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-0/wal/instance
I20260806 07:58:23.427109 5978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:58:23.428448 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37991
--webserver_interface=127.20.202.253
--webserver_port=0
--builtin_ntp_servers=127.20.202.212:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.202.254:36591,127.20.202.253:37991
--flush_threshold_mb=0
--flush_threshold_secs=1
--log_cache_size_limit_mb=1
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_segment_size_mb=1
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--unlock_experimental_flags=true with env {}
I20260806 07:58:23.445407 5978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
W20260806 07:58:23.446772 5925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37991: connect: Connection refused (error 111)
W20260806 07:58:23.448629 5978 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37991: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37991: connect: Connection refused (error 111)
I20260806 07:58:23.498014 5978 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } attempt: 1
W20260806 07:58:23.500399 5978 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37991: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37991: connect: Connection refused (error 111)
I20260806 07:58:23.568713 5978 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } attempt: 2
W20260806 07:58:23.570998 5978 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37991: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37991: connect: Connection refused (error 111)
I20260806 07:58:23.662338 5978 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } attempt: 3
W20260806 07:58:23.664647 5978 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37991: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37991: connect: Connection refused (error 111)
W20260806 07:58:23.784853 5982 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:23.785163 5982 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:23.785243 5982 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:23.793617 5982 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260806 07:58:23.793716 5982 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260806 07:58:23.793788 5982 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 07:58:23.793836 5982 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:23.793891 5982 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 07:58:23.793938 5982 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260806 07:58:23.794021 5982 flags.cc:432] Enabled experimental flag: --flush_threshold_mb=0
W20260806 07:58:23.794062 5982 flags.cc:432] Enabled experimental flag: --flush_threshold_secs=1
I20260806 07:58:23.804340 5982 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.202.254:36591,127.20.202.253:37991
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.202.253:37991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.202.253
--webserver_port=0
--flush_threshold_mb=0
--flush_threshold_secs=1
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--maintenance_manager_num_threads=4
--maintenance_manager_polling_interval_ms=10
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
W20260806 07:58:23.804661 5982 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.202.254:36591,127.20.202.253:37991'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 07:58:23.806043 5982 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:23.807864 5982 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260806 07:58:23.807937 5978 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } attempt: 4
W20260806 07:58:23.810202 5978 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.20.202.253:37991: Network error: Client connection negotiation failed: client connection to 127.20.202.253:37991: connect: Connection refused (error 111)
W20260806 07:58:23.816810 5988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:23.816776 5989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:23.817448 5991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:23.818094 5982 server_base.cc:1061] running on GCE node
I20260806 07:58:23.818864 5982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:23.820295 5982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:23.821530 5982 hybrid_clock.cc:648] HybridClock initialized: now 1786003103821468 us; error 35 us; skew 500 ppm
I20260806 07:58:23.821920 5982 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:58:23.824383 5982 webserver.cc:533] Webserver started at http://127.20.202.253:39327/ using document root <none> and password file <none>
I20260806 07:58:23.825110 5982 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:23.825222 5982 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:23.825536 5982 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:23.828114 5982 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data/instance:
uuid: "15b584d20bab4416a9d975df6675b207"
format_stamp: "Formatted at 2026-08-06 07:58:23 on dist-test-slave-g350"
I20260806 07:58:23.828925 5982 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal/instance:
uuid: "15b584d20bab4416a9d975df6675b207"
format_stamp: "Formatted at 2026-08-06 07:58:23 on dist-test-slave-g350"
I20260806 07:58:23.834753 5982 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 07:58:23.838253 5997 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:23.839668 5982 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 07:58:23.839855 5982 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal
uuid: "15b584d20bab4416a9d975df6675b207"
format_stamp: "Formatted at 2026-08-06 07:58:23 on dist-test-slave-g350"
I20260806 07:58:23.840087 5982 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:23.862987 5982 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:23.863874 5982 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:23.864177 5982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:23.886759 5982 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.253:37991
I20260806 07:58:23.886788 6051 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.253:37991 every 8 connection(s)
I20260806 07:58:23.888391 5982 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/data/info.pb
I20260806 07:58:23.892735 6052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:23.894811 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 5982
I20260806 07:58:23.895046 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/master-1/wal/instance
I20260806 07:58:23.901031 6052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:58:23.912647 6052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
I20260806 07:58:23.922295 6052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Bootstrap starting.
I20260806 07:58:23.925539 6052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:23.926671 6052 log.cc:826] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:23.929797 6052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: No bootstrap required, opened a new log
I20260806 07:58:23.935670 6052 raft_consensus.cc:359] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
I20260806 07:58:23.936021 6052 raft_consensus.cc:385] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:23.936136 6052 raft_consensus.cc:740] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b584d20bab4416a9d975df6675b207, State: Initialized, Role: FOLLOWER
I20260806 07:58:23.936787 6052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
I20260806 07:58:23.937700 6059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } } }
I20260806 07:58:23.937976 6059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:58:23.938805 6052 sys_catalog.cc:565] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 07:58:23.949481 6006 tablet.cc:2406] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:58:23.954613 6070 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:58:23.954757 6070 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:58:23.950917 5982 master_runner.cc:187] Error getting master registration for 127.20.202.254:36591: OK, instance_id { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" instance_seqno: 1786003103408150 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 07:58:23.956821 5982 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260806 07:58:23.978199 6059 raft_consensus.cc:493] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:23.978538 6059 raft_consensus.cc:515] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
I20260806 07:58:23.980183 6059 leader_election.cc:290] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d2d029e02f42bdb166bf54079a7128 (127.20.202.254:36591)
I20260806 07:58:24.109457 5978 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } attempt: 5
I20260806 07:58:24.125687 5978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Bootstrap starting.
I20260806 07:58:24.129990 5978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:24.131346 5978 log.cc:826] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:24.134588 5978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: No bootstrap required, opened a new log
I20260806 07:58:24.142339 5978 raft_consensus.cc:359] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
I20260806 07:58:24.142702 5978 raft_consensus.cc:385] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:24.142818 5978 raft_consensus.cc:740] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d2d029e02f42bdb166bf54079a7128, State: Initialized, Role: FOLLOWER
I20260806 07:58:24.143548 5978 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
I20260806 07:58:24.144568 6073 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } } }
I20260806 07:58:24.144936 6073 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:58:24.145934 5978 sys_catalog.cc:565] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 07:58:24.146037 5953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15b584d20bab4416a9d975df6675b207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d2d029e02f42bdb166bf54079a7128" is_pre_election: true
I20260806 07:58:24.146552 5953 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15b584d20bab4416a9d975df6675b207 in term 0.
I20260806 07:58:24.147251 5999 leader_election.cc:304] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 15b584d20bab4416a9d975df6675b207, a3d2d029e02f42bdb166bf54079a7128; no voters:
I20260806 07:58:24.147933 6059 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:24.148095 6059 raft_consensus.cc:493] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:24.148211 6059 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:58:24.148543 5932 tablet.cc:2406] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:58:24.151623 6059 raft_consensus.cc:515] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
I20260806 07:58:24.152285 6059 leader_election.cc:290] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [CANDIDATE]: Term 1 election: Requested vote from peers a3d2d029e02f42bdb166bf54079a7128 (127.20.202.254:36591)
I20260806 07:58:24.152998 5953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15b584d20bab4416a9d975df6675b207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d2d029e02f42bdb166bf54079a7128"
I20260806 07:58:24.153261 5953 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:24.157756 5953 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15b584d20bab4416a9d975df6675b207 in term 1.
I20260806 07:58:24.158461 5999 leader_election.cc:304] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 15b584d20bab4416a9d975df6675b207, a3d2d029e02f42bdb166bf54079a7128; no voters:
I20260806 07:58:24.158960 6059 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:24.159751 6059 raft_consensus.cc:697] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [term 1 LEADER]: Becoming Leader. State: Replica: 15b584d20bab4416a9d975df6675b207, State: Running, Role: LEADER
I20260806 07:58:24.160425 6059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } }
W20260806 07:58:24.166357 6085 catalog_manager.cc:1594] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 07:58:24.166757 6085 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 07:58:24.168918 6084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15b584d20bab4416a9d975df6675b207. Latest consensus state: current_term: 1 leader_uuid: "15b584d20bab4416a9d975df6675b207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } } }
I20260806 07:58:24.169250 6084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:24.170709 6088 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260806 07:58:24.173177 6088 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260806 07:58:24.182423 5953 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [term 1 FOLLOWER]: Refusing update from remote peer 15b584d20bab4416a9d975df6675b207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 07:58:24.183450 6084 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:24.195006 6073 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15b584d20bab4416a9d975df6675b207. Latest consensus state: current_term: 1 leader_uuid: "15b584d20bab4416a9d975df6675b207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } } }
I20260806 07:58:24.195256 6073 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:58:24.206128 6084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15b584d20bab4416a9d975df6675b207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } } }
I20260806 07:58:24.206393 6084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207 [sys.catalog]: This master's current role is: LEADER
I20260806 07:58:24.208130 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:24.209868 6088 catalog_manager.cc:1383] Generated new cluster ID: 284017fb075041078160946b3a7ce881
I20260806 07:58:24.210004 6088 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260806 07:58:24.222982 6073 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15b584d20bab4416a9d975df6675b207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d2d029e02f42bdb166bf54079a7128" member_type: VOTER last_known_addr { host: "127.20.202.254" port: 36591 } } peers { permanent_uuid: "15b584d20bab4416a9d975df6675b207" member_type: VOTER last_known_addr { host: "127.20.202.253" port: 37991 } } }
I20260806 07:58:24.223209 6073 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 07:58:24.223855 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:24.259956 6088 catalog_manager.cc:1406] Generated new certificate authority record
I20260806 07:58:24.261752 6088 catalog_manager.cc:1540] Loading token signing keys...
I20260806 07:58:24.288196 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:24.288653 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.290373 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260806 07:58:24.292996 6088 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Generated new TSK 0
I20260806 07:58:24.294142 6088 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260806 07:58:24.279242 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.070s user 0.029s sys 0.016s Metrics: {"bytes_written":4776,"cfile_init":1,"dirs.queue_time_us":56,"dirs.run_cpu_time_us":244,"dirs.run_wall_time_us":1040,"drs_written":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1859,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1}
I20260806 07:58:24.305485 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.081s user 0.026s sys 0.008s Metrics: {"bytes_written":4776,"cfile_init":1,"dirs.queue_time_us":8853,"dirs.run_cpu_time_us":226,"dirs.run_wall_time_us":5436,"drs_written":1,"lbm_read_time_us":123,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1029,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1,"thread_start_us":9055,"threads_started":1}
I20260806 07:58:24.307020 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:24.307605 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.193:0
--local_ip_for_outbound_sockets=127.20.202.193
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36591,127.20.202.253:37991
--builtin_ntp_servers=127.20.202.212:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:58:24.307751 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.308091 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.308360 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.312523 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:24.313583 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:24.314929 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.013s sys 0.000s Metrics: {"bytes_written":5617,"cfile_init":1,"dirs.queue_time_us":54,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":1838,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":959,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1}
I20260806 07:58:24.319852 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:24.320237 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.320569 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.320850 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.323770 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":99,"lbm_reads_lt_1ms":4}
I20260806 07:58:24.324323 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:24.324642 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:24.325649 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4}
I20260806 07:58:24.334172 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.015s sys 0.000s Metrics: {"bytes_written":5966,"cfile_init":1,"dirs.queue_time_us":289,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":1820,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1018,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":2}
I20260806 07:58:24.336133 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 174 bytes on disk
I20260806 07:58:24.336573 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260806 07:58:24.337435 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014997
I20260806 07:58:24.337338 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4}
I20260806 07:58:24.348325 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.012s sys 0.000s Metrics: {"bytes_written":5068,"cfile_init":1,"dirs.queue_time_us":53,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":946,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":980,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":1}
I20260806 07:58:24.348553 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.349084 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 163 bytes on disk
I20260806 07:58:24.350064 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":113,"lbm_reads_lt_1ms":4}
I20260806 07:58:24.350428 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:24.370941 5931 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:24.377023 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.013s sys 0.004s Metrics: {"dirs.queue_time_us":1994,"dirs.run_cpu_time_us":1212,"dirs.run_wall_time_us":10583,"lbm_write_time_us":441,"lbm_writes_lt_1ms":7,"mutex_wait_us":170,"num_input_rowsets":0,"thread_start_us":22980,"threads_started":2}
I20260806 07:58:24.388379 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260806 07:58:24.388801 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 185 bytes on disk
I20260806 07:58:24.389213 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:24.389215 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4}
I20260806 07:58:24.396560 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.058s user 0.043s sys 0.000s Metrics: {"bytes_written":6026,"cfile_cache_hit":12,"cfile_cache_hit_bytes":1552,"cfile_cache_miss":28,"cfile_cache_miss_bytes":2194,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":1560,"dirs.run_cpu_time_us":677,"dirs.run_wall_time_us":3652,"drs_written":1,"lbm_read_time_us":1296,"lbm_reads_lt_1ms":56,"lbm_write_time_us":1460,"lbm_writes_lt_1ms":27,"mutex_wait_us":36,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":3,"spinlock_wait_cycles":17152,"thread_start_us":7629,"threads_started":1}
W20260806 07:58:24.751165 6095 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:24.751480 6095 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:24.751648 6095 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:24.760468 6095 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:24.760723 6095 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.193
I20260806 07:58:24.771828 6095 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.202.193
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36591,127.20.202.253:37991
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:24.773478 6095 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:24.775427 6095 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:24.786208 6104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:24.786159 6103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:24.787248 6106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:24.787784 6095 server_base.cc:1061] running on GCE node
I20260806 07:58:24.788385 6095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:24.789443 6095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:24.790680 6095 hybrid_clock.cc:648] HybridClock initialized: now 1786003104790620 us; error 45 us; skew 500 ppm
I20260806 07:58:24.791028 6095 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:58:24.793380 6095 webserver.cc:533] Webserver started at http://127.20.202.193:33017/ using document root <none> and password file <none>
I20260806 07:58:24.794189 6095 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:24.794319 6095 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:24.794685 6095 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:24.797154 6095 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data/instance:
uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
format_stamp: "Formatted at 2026-08-06 07:58:24 on dist-test-slave-g350"
I20260806 07:58:24.798024 6095 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance:
uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
format_stamp: "Formatted at 2026-08-06 07:58:24 on dist-test-slave-g350"
I20260806 07:58:24.803936 6095 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 07:58:24.807871 6112 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:24.809399 6095 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:24.809665 6095 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal
uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
format_stamp: "Formatted at 2026-08-06 07:58:24 on dist-test-slave-g350"
I20260806 07:58:24.809913 6095 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:24.825640 6095 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:24.826566 6095 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:24.826867 6095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:24.827917 6095 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:24.829767 6095 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:24.829900 6095 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:24.830034 6095 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:24.830121 6095 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:24.867748 6095 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.193:45079
I20260806 07:58:24.867839 6225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.193:45079 every 8 connection(s)
I20260806 07:58:24.869446 6095 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/data/info.pb
I20260806 07:58:24.876271 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6095
I20260806 07:58:24.876490 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/instance
I20260806 07:58:24.880368 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.194:0
--local_ip_for_outbound_sockets=127.20.202.194
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36591,127.20.202.253:37991
--builtin_ntp_servers=127.20.202.212:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:58:24.885488 6226 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37991
I20260806 07:58:24.885720 6227 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36591
I20260806 07:58:24.885941 6226 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:24.885938 6227 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:24.887193 6226 heartbeater.cc:507] Master 127.20.202.253:37991 requested a full tablet report, sending...
I20260806 07:58:24.887247 6227 heartbeater.cc:507] Master 127.20.202.254:36591 requested a full tablet report, sending...
I20260806 07:58:24.889796 6016 ts_manager.cc:194] Registered new tserver with Master: 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079)
I20260806 07:58:24.889870 5943 ts_manager.cc:194] Registered new tserver with Master: 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079)
I20260806 07:58:24.892011 6016 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.193:57297
I20260806 07:58:25.172135 6085 catalog_manager.cc:1295] Loaded cluster ID: 284017fb075041078160946b3a7ce881
I20260806 07:58:25.172255 6085 catalog_manager.cc:1588] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: loading cluster ID for follower catalog manager: success
I20260806 07:58:25.177062 6085 catalog_manager.cc:1610] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: acquiring CA information for follower catalog manager: success
I20260806 07:58:25.180526 6085 catalog_manager.cc:1638] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 07:58:25.245803 6233 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:25.246131 6233 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:25.246243 6233 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:25.254776 6233 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:25.254930 6233 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.194
I20260806 07:58:25.265038 6233 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.202.194
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36591,127.20.202.253:37991
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:25.266619 6233 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:25.268358 6233 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:25.277999 6239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:25.278707 6238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:25.278787 6241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:25.279460 6233 server_base.cc:1061] running on GCE node
I20260806 07:58:25.280082 6233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:25.281266 6233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:25.282507 6233 hybrid_clock.cc:648] HybridClock initialized: now 1786003105282446 us; error 57 us; skew 500 ppm
I20260806 07:58:25.282843 6233 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:58:25.285132 6233 webserver.cc:533] Webserver started at http://127.20.202.194:46359/ using document root <none> and password file <none>
I20260806 07:58:25.285979 6233 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:25.286127 6233 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:25.286504 6233 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:25.289460 6233 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data/instance:
uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
format_stamp: "Formatted at 2026-08-06 07:58:25 on dist-test-slave-g350"
I20260806 07:58:25.290591 6233 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance:
uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
format_stamp: "Formatted at 2026-08-06 07:58:25 on dist-test-slave-g350"
I20260806 07:58:25.296664 6233 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 07:58:25.300271 6247 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:25.301710 6233 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:25.301906 6233 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal
uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
format_stamp: "Formatted at 2026-08-06 07:58:25 on dist-test-slave-g350"
I20260806 07:58:25.302203 6233 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:25.323458 6233 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:25.324406 6233 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:25.324713 6233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:25.325851 6233 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:25.327606 6233 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:25.327746 6233 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:25.327874 6233 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:25.327966 6233 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:25.366609 6233 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.194:32979
I20260806 07:58:25.366698 6363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.194:32979 every 8 connection(s)
I20260806 07:58:25.368112 6233 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/data/info.pb
I20260806 07:58:25.373693 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6233
I20260806 07:58:25.373903 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/instance
I20260806 07:58:25.378252 21291 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskUU61cc/build/asan/bin/kudu
/tmp/dist-test-taskUU61cc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.202.195:0
--local_ip_for_outbound_sockets=127.20.202.195
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36591,127.20.202.253:37991
--builtin_ntp_servers=127.20.202.212:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260806 07:58:25.383541 6365 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36591
I20260806 07:58:25.383697 6364 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37991
I20260806 07:58:25.383934 6365 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:25.383939 6364 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:25.385118 6364 heartbeater.cc:507] Master 127.20.202.253:37991 requested a full tablet report, sending...
I20260806 07:58:25.385190 6365 heartbeater.cc:507] Master 127.20.202.254:36591 requested a full tablet report, sending...
I20260806 07:58:25.386924 6016 ts_manager.cc:194] Registered new tserver with Master: 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:25.387023 5943 ts_manager.cc:194] Registered new tserver with Master: 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:25.388013 6016 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.194:36619
W20260806 07:58:25.742168 6369 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 07:58:25.742575 6369 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 07:58:25.742727 6369 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260806 07:58:25.752550 6369 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260806 07:58:25.752802 6369 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.202.195
I20260806 07:58:25.764273 6369 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.202.212:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.202.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.202.195
--webserver_port=0
--tserver_master_addrs=127.20.202.254:36591,127.20.202.253:37991
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.202.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision cb747ff0ca0209e3fa98be4167c67edd474acd67
build type FASTDEBUG
built by None at 06 Aug 2026 07:43:24 UTC on bdcb31816ec0
build id 13206
ASAN enabled
I20260806 07:58:25.766023 6369 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 07:58:25.767972 6369 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260806 07:58:25.778815 6374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:25.778776 6375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 07:58:25.779784 6377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 07:58:25.780129 6369 server_base.cc:1061] running on GCE node
I20260806 07:58:25.780821 6369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 07:58:25.782037 6369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 07:58:25.783267 6369 hybrid_clock.cc:648] HybridClock initialized: now 1786003105783216 us; error 36 us; skew 500 ppm
I20260806 07:58:25.783627 6369 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 07:58:25.786058 6369 webserver.cc:533] Webserver started at http://127.20.202.195:40559/ using document root <none> and password file <none>
I20260806 07:58:25.786851 6369 fs_manager.cc:362] Metadata directory not provided
I20260806 07:58:25.786948 6369 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 07:58:25.787345 6369 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 07:58:25.789951 6369 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data/instance:
uuid: "5d4b117032604585af3544cb030a10f2"
format_stamp: "Formatted at 2026-08-06 07:58:25 on dist-test-slave-g350"
I20260806 07:58:25.790807 6369 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance:
uuid: "5d4b117032604585af3544cb030a10f2"
format_stamp: "Formatted at 2026-08-06 07:58:25 on dist-test-slave-g350"
I20260806 07:58:25.796557 6369 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 07:58:25.800640 6383 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:25.802386 6369 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:25.802529 6369 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal
uuid: "5d4b117032604585af3544cb030a10f2"
format_stamp: "Formatted at 2026-08-06 07:58:25 on dist-test-slave-g350"
I20260806 07:58:25.802714 6369 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 07:58:25.821446 6369 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260806 07:58:25.822423 6369 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 07:58:25.822713 6369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 07:58:25.823793 6369 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260806 07:58:25.825452 6369 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 07:58:25.825542 6369 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:25.825670 6369 ts_tablet_manager.cc:616] Registered 0 tablets
I20260806 07:58:25.825733 6369 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 07:58:25.863953 6369 rpc_server.cc:307] RPC server started. Bound to: 127.20.202.195:36665
I20260806 07:58:25.864094 6497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.20.202.195:36665 every 8 connection(s)
I20260806 07:58:25.865746 6369 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/data/info.pb
I20260806 07:58:25.874851 21291 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskUU61cc/build/asan/bin/kudu as pid 6369
I20260806 07:58:25.875063 21291 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/instance
I20260806 07:58:25.882283 6498 heartbeater.cc:344] Connected to a master server at 127.20.202.253:37991
I20260806 07:58:25.882697 6498 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:25.883564 6498 heartbeater.cc:507] Master 127.20.202.253:37991 requested a full tablet report, sending...
I20260806 07:58:25.884629 6499 heartbeater.cc:344] Connected to a master server at 127.20.202.254:36591
I20260806 07:58:25.884809 6499 heartbeater.cc:461] Registering TS with master...
I20260806 07:58:25.885185 6016 ts_manager.cc:194] Registered new tserver with Master: 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:25.885468 6499 heartbeater.cc:507] Master 127.20.202.254:36591 requested a full tablet report, sending...
I20260806 07:58:25.886231 6016 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.20.202.195:39915
I20260806 07:58:25.886885 5943 ts_manager.cc:194] Registered new tserver with Master: 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:25.895087 6226 heartbeater.cc:499] Master 127.20.202.253:37991 was elected leader, sending a full tablet report...
I20260806 07:58:25.897310 21291 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260806 07:58:25.923707 6016 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46090:
name: "table-0"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
W20260806 07:58:25.925734 6016 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 07:58:25.938427 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:25.957888 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:25.964119 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.019s sys 0.000s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":8811,"dirs.run_cpu_time_us":258,"dirs.run_wall_time_us":1592,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":747,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17,"thread_start_us":8662,"threads_started":1}
I20260806 07:58:25.965570 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.032994
I20260806 07:58:25.966367 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.032994
I20260806 07:58:25.972407 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.032994
I20260806 07:58:25.975132 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:25.996305 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.014s sys 0.000s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":402,"dirs.run_cpu_time_us":230,"dirs.run_wall_time_us":21982,"drs_written":1,"lbm_read_time_us":100,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1220,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17,"thread_start_us":292,"threads_started":1}
I20260806 07:58:25.997915 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:25.998344 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:25.998708 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.000772 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.005s sys 0.000s Metrics: {"bytes_written":684,"delete_count":0,"lbm_write_time_us":132,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":14}
I20260806 07:58:26.001603 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.003016 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":131,"lbm_reads_lt_1ms":4}
I20260806 07:58:26.003412 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.004594 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.014994
I20260806 07:58:26.017305 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.003s sys 0.004s Metrics: {"bytes_written":716,"delete_count":0,"lbm_write_time_us":274,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260806 07:58:26.041929 6005 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:26.048569 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.046s user 0.006s sys 0.000s Metrics: {"bytes_written":448,"delete_count":0,"lbm_write_time_us":9780,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":3,"reinsert_count":0,"update_count":2}
I20260806 07:58:26.059561 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 639 bytes on disk
I20260806 07:58:26.059871 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 639 bytes on disk
I20260806 07:58:26.060165 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 639 bytes on disk
I20260806 07:58:26.060819 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4}
I20260806 07:58:26.061259 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.061537 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.064188 6433 tablet_service.cc:1511] Processing CreateTablet for tablet 6047b73f65e94f58b985a178aec1e568 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.064931 6289 tablet_service.cc:1511] Processing CreateTablet for tablet 8d6639b2db4f4cb99f33465ace8f0294 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.065910 6433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6047b73f65e94f58b985a178aec1e568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.066157 6289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d6639b2db4f4cb99f33465ace8f0294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.064708 6293 tablet_service.cc:1511] Processing CreateTablet for tablet f59dfcb1b67340e2b93fc9e8644df607 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.067625 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59dfcb1b67340e2b93fc9e8644df607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.067884 6291 tablet_service.cc:1511] Processing CreateTablet for tablet 1d800927238145aea3e3c82d1bf158e1 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.068488 6291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d800927238145aea3e3c82d1bf158e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.069396 6287 tablet_service.cc:1511] Processing CreateTablet for tablet 0d04186bb02946928cc22152266d8a3f (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.070001 6287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d04186bb02946928cc22152266d8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.070595 6285 tablet_service.cc:1511] Processing CreateTablet for tablet fb7a45d28865477bbe3cef826d477a93 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.071039 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7a45d28865477bbe3cef826d477a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.074380 6432 tablet_service.cc:1511] Processing CreateTablet for tablet 6305d69bfe42420a92ca7a9fc36ec2f6 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.074967 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6305d69bfe42420a92ca7a9fc36ec2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.075011 6292 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7d632b15694509ac4cce794498fbb3 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.075132 6428 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7d632b15694509ac4cce794498fbb3 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.075598 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7d632b15694509ac4cce794498fbb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.075636 6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7d632b15694509ac4cce794498fbb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.076385 6288 tablet_service.cc:1511] Processing CreateTablet for tablet ff476732379d406e8093e43da857d681 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.076864 6290 tablet_service.cc:1511] Processing CreateTablet for tablet 727a2816995c4456b2210e7ad8913e3c (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.077205 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff476732379d406e8093e43da857d681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.078035 6429 tablet_service.cc:1511] Processing CreateTablet for tablet f59dfcb1b67340e2b93fc9e8644df607 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.078490 6429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59dfcb1b67340e2b93fc9e8644df607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.078594 6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 727a2816995c4456b2210e7ad8913e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.074399 6430 tablet_service.cc:1511] Processing CreateTablet for tablet 1f7851b0e26d40e48bd60a7671b643b9 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.084532 6430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f7851b0e26d40e48bd60a7671b643b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.085664 6295 tablet_service.cc:1511] Processing CreateTablet for tablet 6305d69bfe42420a92ca7a9fc36ec2f6 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.086319 6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6305d69bfe42420a92ca7a9fc36ec2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.086896 6426 tablet_service.cc:1511] Processing CreateTablet for tablet 727a2816995c4456b2210e7ad8913e3c (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.086990 6425 tablet_service.cc:1511] Processing CreateTablet for tablet 8d6639b2db4f4cb99f33465ace8f0294 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.087495 6003 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:26.090363 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 278 bytes on disk
I20260806 07:58:26.091991 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.119s user 0.014s sys 0.002s Metrics: {"dirs.queue_time_us":39819,"dirs.run_cpu_time_us":1783,"dirs.run_wall_time_us":11273,"lbm_write_time_us":323,"lbm_writes_lt_1ms":7,"mutex_wait_us":1370,"num_input_rowsets":0,"spinlock_wait_cycles":5893760,"thread_start_us":81979,"threads_started":2}
I20260806 07:58:26.092768 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.219296
I20260806 07:58:26.092660 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4}
I20260806 07:58:26.093362 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.219296
I20260806 07:58:26.095114 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.096285 6296 tablet_service.cc:1511] Processing CreateTablet for tablet 6047b73f65e94f58b985a178aec1e568 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.097038 6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6047b73f65e94f58b985a178aec1e568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.097716 6284 tablet_service.cc:1511] Processing CreateTablet for tablet 523ed5cfaa914101a9946b4391aad3c0 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.098449 6283 tablet_service.cc:1511] Processing CreateTablet for tablet 1bce3ad6757940269b58768139f10bbf (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.064607 6286 tablet_service.cc:1511] Processing CreateTablet for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.099274 6282 tablet_service.cc:1511] Processing CreateTablet for tablet cbcae8fb644044d99a96dd9fb716965c (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.099663 6281 tablet_service.cc:1511] Processing CreateTablet for tablet b6d0ae6d095640eea8caa5853c1fbb2a (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.102072 6284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523ed5cfaa914101a9946b4391aad3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.102303 6283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bce3ad6757940269b58768139f10bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.102541 6282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcae8fb644044d99a96dd9fb716965c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.102787 6281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d0ae6d095640eea8caa5853c1fbb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.103018 6286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.053506 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.083s user 0.006s sys 0.003s Metrics: {"dirs.queue_time_us":2762,"dirs.run_cpu_time_us":916,"dirs.run_wall_time_us":14051,"lbm_write_time_us":10174,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":6,"mutex_wait_us":6575,"num_input_rowsets":0,"spinlock_wait_cycles":16050944,"thread_start_us":6054,"threads_started":1}
I20260806 07:58:26.121483 6158 tablet_service.cc:1511] Processing CreateTablet for tablet 1f7851b0e26d40e48bd60a7671b643b9 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.121853 6152 tablet_service.cc:1511] Processing CreateTablet for tablet ff476732379d406e8093e43da857d681 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.122619 6156 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7d632b15694509ac4cce794498fbb3 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.122898 6152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff476732379d406e8093e43da857d681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.123404 6155 tablet_service.cc:1511] Processing CreateTablet for tablet 1d800927238145aea3e3c82d1bf158e1 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.123878 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d800927238145aea3e3c82d1bf158e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.124886 6153 tablet_service.cc:1511] Processing CreateTablet for tablet 8d6639b2db4f4cb99f33465ace8f0294 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.125350 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d6639b2db4f4cb99f33465ace8f0294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.125943 6158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f7851b0e26d40e48bd60a7671b643b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.126377 6151 tablet_service.cc:1511] Processing CreateTablet for tablet 0d04186bb02946928cc22152266d8a3f (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.121119 6149 tablet_service.cc:1511] Processing CreateTablet for tablet fb7a45d28865477bbe3cef826d477a93 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.126830 6151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d04186bb02946928cc22152266d8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.127363 6150 tablet_service.cc:1511] Processing CreateTablet for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.128264 6159 tablet_service.cc:1511] Processing CreateTablet for tablet 6305d69bfe42420a92ca7a9fc36ec2f6 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.128626 6154 tablet_service.cc:1511] Processing CreateTablet for tablet 727a2816995c4456b2210e7ad8913e3c (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.128991 6160 tablet_service.cc:1511] Processing CreateTablet for tablet 6047b73f65e94f58b985a178aec1e568 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.129333 6156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7d632b15694509ac4cce794498fbb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.131434 6157 tablet_service.cc:1511] Processing CreateTablet for tablet f59dfcb1b67340e2b93fc9e8644df607 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.132021 6150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.133345 6159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6305d69bfe42420a92ca7a9fc36ec2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.133829 6294 tablet_service.cc:1511] Processing CreateTablet for tablet 1f7851b0e26d40e48bd60a7671b643b9 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.134073 6154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 727a2816995c4456b2210e7ad8913e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.134325 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f7851b0e26d40e48bd60a7671b643b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.134426 6148 tablet_service.cc:1511] Processing CreateTablet for tablet 523ed5cfaa914101a9946b4391aad3c0 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.134634 6147 tablet_service.cc:1511] Processing CreateTablet for tablet 1bce3ad6757940269b58768139f10bbf (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.135414 6532 tablet_bootstrap.cc:492] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.139685 6427 tablet_service.cc:1511] Processing CreateTablet for tablet 1d800927238145aea3e3c82d1bf158e1 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.146431 6426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 727a2816995c4456b2210e7ad8913e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.149009 6146 tablet_service.cc:1511] Processing CreateTablet for tablet cbcae8fb644044d99a96dd9fb716965c (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.149832 6425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d6639b2db4f4cb99f33465ace8f0294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.151882 6424 tablet_service.cc:1511] Processing CreateTablet for tablet ff476732379d406e8093e43da857d681 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.152423 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff476732379d406e8093e43da857d681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.153843 6427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d800927238145aea3e3c82d1bf158e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.157799 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6047b73f65e94f58b985a178aec1e568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.163132 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523ed5cfaa914101a9946b4391aad3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.165706 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.166265 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.168886 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bce3ad6757940269b58768139f10bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.175298 6422 tablet_service.cc:1511] Processing CreateTablet for tablet 1bce3ad6757940269b58768139f10bbf (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.175308 6421 tablet_service.cc:1511] Processing CreateTablet for tablet cbcae8fb644044d99a96dd9fb716965c (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.175913 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bce3ad6757940269b58768139f10bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.176152 6421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcae8fb644044d99a96dd9fb716965c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.179688 6157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59dfcb1b67340e2b93fc9e8644df607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.182401 6145 tablet_service.cc:1511] Processing CreateTablet for tablet b6d0ae6d095640eea8caa5853c1fbb2a (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.182849 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7a45d28865477bbe3cef826d477a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.181885 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.215s user 0.030s sys 0.012s Metrics: {"bytes_written":716,"cfile_cache_hit":24,"cfile_cache_hit_bytes":3391,"cfile_cache_miss":58,"cfile_cache_miss_bytes":5466,"cfile_init":15,"delete_count":0,"delta_iterators_relevant":9,"dirs.queue_time_us":52484,"dirs.run_cpu_time_us":1568,"dirs.run_wall_time_us":72957,"drs_written":1,"lbm_read_time_us":41443,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":116,"lbm_write_time_us":1415,"lbm_writes_lt_1ms":31,"mutex_wait_us":46105,"num_input_rowsets":4,"peak_mem_usage":131072,"reinsert_count":0,"rows_written":20,"spinlock_wait_cycles":17038080,"thread_start_us":40646,"threads_started":1,"update_count":16}
I20260806 07:58:26.184892 6145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d0ae6d095640eea8caa5853c1fbb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.186825 6532 tablet_bootstrap.cc:654] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.189098 6532 log.cc:826] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:26.190984 6432 tablet_service.cc:1511] Processing CreateTablet for tablet 523ed5cfaa914101a9946b4391aad3c0 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.191561 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523ed5cfaa914101a9946b4391aad3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.191872 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcae8fb644044d99a96dd9fb716965c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.193256 6424 tablet_service.cc:1511] Processing CreateTablet for tablet 0d04186bb02946928cc22152266d8a3f (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.193670 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d04186bb02946928cc22152266d8a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.195547 6423 tablet_service.cc:1511] Processing CreateTablet for tablet fb7a45d28865477bbe3cef826d477a93 (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.195942 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7a45d28865477bbe3cef826d477a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.197276 6429 tablet_service.cc:1511] Processing CreateTablet for tablet b6d0ae6d095640eea8caa5853c1fbb2a (DEFAULT_TABLE table=table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:26.198168 6429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d0ae6d095640eea8caa5853c1fbb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:26.213537 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.205s user 0.064s sys 0.065s Metrics: {"bytes_written":12022,"cfile_cache_hit":23,"cfile_cache_hit_bytes":4719,"cfile_cache_miss":19,"cfile_cache_miss_bytes":3022,"cfile_init":5,"delta_iterators_relevant":5,"dirs.queue_time_us":2088,"dirs.run_cpu_time_us":1008,"dirs.run_wall_time_us":32341,"drs_written":1,"lbm_read_time_us":1265,"lbm_reads_lt_1ms":39,"lbm_write_time_us":1262,"lbm_writes_lt_1ms":27,"mutex_wait_us":569,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":20,"thread_start_us":154404,"threads_started":3}
I20260806 07:58:26.219502 6532 tablet_bootstrap.cc:492] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.219844 6532 ts_tablet_manager.cc:1403] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.085s user 0.003s sys 0.011s
I20260806 07:58:26.221441 6531 tablet_bootstrap.cc:492] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.227612 6532 raft_consensus.cc:359] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.228040 6532 raft_consensus.cc:385] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.228140 6532 raft_consensus.cc:740] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.229027 6532 consensus_queue.cc:260] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.237438 6532 ts_tablet_manager.cc:1434] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20260806 07:58:26.237901 6364 heartbeater.cc:499] Master 127.20.202.253:37991 was elected leader, sending a full tablet report...
I20260806 07:58:26.242422 6537 tablet_bootstrap.cc:492] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.246726 6531 tablet_bootstrap.cc:654] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.248521 6531 log.cc:826] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:26.250174 6537 tablet_bootstrap.cc:654] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.251698 6532 tablet_bootstrap.cc:492] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.251786 6537 log.cc:826] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Log is configured to *not* fsync() on all Append() calls
I20260806 07:58:26.255798 6532 tablet_bootstrap.cc:654] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.259229 6531 tablet_bootstrap.cc:492] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.259557 6531 ts_tablet_manager.cc:1403] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.004s
I20260806 07:58:26.259595 6532 tablet_bootstrap.cc:492] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.259742 6532 ts_tablet_manager.cc:1403] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260806 07:58:26.260769 6532 raft_consensus.cc:359] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.260984 6532 raft_consensus.cc:385] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.261085 6532 raft_consensus.cc:740] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.261358 6532 consensus_queue.cc:260] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.261916 6532 ts_tablet_manager.cc:1434] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 07:58:26.262235 6532 tablet_bootstrap.cc:492] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.264010 6532 tablet_bootstrap.cc:654] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.264580 6537 tablet_bootstrap.cc:492] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.264894 6537 ts_tablet_manager.cc:1403] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.001s
I20260806 07:58:26.274453 6537 raft_consensus.cc:359] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.274930 6537 raft_consensus.cc:385] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.275069 6537 raft_consensus.cc:740] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.275870 6537 consensus_queue.cc:260] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.285616 6531 raft_consensus.cc:359] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.287262 6531 raft_consensus.cc:385] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.287472 6531 raft_consensus.cc:740] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.288842 6531 consensus_queue.cc:260] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.290948 6537 ts_tablet_manager.cc:1434] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.026s user 0.013s sys 0.011s
I20260806 07:58:26.295425 6532 tablet_bootstrap.cc:492] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.295645 6532 ts_tablet_manager.cc:1403] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.019s
I20260806 07:58:26.295495 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.202s user 0.015s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2932,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":95,"lbm_read_time_us":185177,"lbm_reads_gt_100_ms":1,"lbm_reads_lt_1ms":20,"lbm_write_time_us":382,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":16}
I20260806 07:58:26.296334 6537 tablet_bootstrap.cc:492] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.296521 6532 raft_consensus.cc:359] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.296713 6532 raft_consensus.cc:385] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.296780 6532 raft_consensus.cc:740] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.297092 6532 consensus_queue.cc:260] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.297680 6532 ts_tablet_manager.cc:1434] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:26.298027 6532 tablet_bootstrap.cc:492] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.298276 6537 tablet_bootstrap.cc:654] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.300432 6532 tablet_bootstrap.cc:654] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.306236 6498 heartbeater.cc:499] Master 127.20.202.253:37991 was elected leader, sending a full tablet report...
I20260806 07:58:26.307106 6531 ts_tablet_manager.cc:1434] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.047s user 0.018s sys 0.004s
I20260806 07:58:26.307642 6531 tablet_bootstrap.cc:492] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.310109 6531 tablet_bootstrap.cc:654] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.312568 6537 tablet_bootstrap.cc:492] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.312783 6537 ts_tablet_manager.cc:1403] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20260806 07:58:26.313799 6532 tablet_bootstrap.cc:492] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.314021 6532 ts_tablet_manager.cc:1403] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260806 07:58:26.314054 6537 raft_consensus.cc:359] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.314286 6537 raft_consensus.cc:385] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.314385 6537 raft_consensus.cc:740] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.314455 6531 tablet_bootstrap.cc:492] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.314724 6531 ts_tablet_manager.cc:1403] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 07:58:26.314672 6537 consensus_queue.cc:260] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.315320 6532 raft_consensus.cc:359] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.315582 6532 raft_consensus.cc:385] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.315749 6532 raft_consensus.cc:740] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.315773 6531 raft_consensus.cc:359] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.315975 6531 raft_consensus.cc:385] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.316077 6531 raft_consensus.cc:740] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.316236 6537 ts_tablet_manager.cc:1434] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:26.316268 6532 consensus_queue.cc:260] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.316421 6531 consensus_queue.cc:260] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.316525 6537 tablet_bootstrap.cc:492] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.316959 6532 ts_tablet_manager.cc:1434] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:26.317024 6531 ts_tablet_manager.cc:1434] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:26.317324 6531 tablet_bootstrap.cc:492] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.317349 6532 tablet_bootstrap.cc:492] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.318459 6537 tablet_bootstrap.cc:654] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.319103 6531 tablet_bootstrap.cc:654] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.319406 6532 tablet_bootstrap.cc:654] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.324756 6532 tablet_bootstrap.cc:492] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.325011 6532 ts_tablet_manager.cc:1403] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 07:58:26.326025 6537 tablet_bootstrap.cc:492] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.325953 6532 raft_consensus.cc:359] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.326239 6537 ts_tablet_manager.cc:1403] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 07:58:26.326332 6532 raft_consensus.cc:385] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.326481 6532 raft_consensus.cc:740] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.326839 6532 consensus_queue.cc:260] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.327399 6532 ts_tablet_manager.cc:1434] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 07:58:26.327507 6537 raft_consensus.cc:359] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.327715 6537 raft_consensus.cc:385] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.327769 6532 tablet_bootstrap.cc:492] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.327800 6537 raft_consensus.cc:740] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.328071 6537 consensus_queue.cc:260] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.328640 6537 ts_tablet_manager.cc:1434] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:26.328961 6537 tablet_bootstrap.cc:492] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.329716 6532 tablet_bootstrap.cc:654] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.330469 6531 tablet_bootstrap.cc:492] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.330742 6531 ts_tablet_manager.cc:1403] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260806 07:58:26.331164 6537 tablet_bootstrap.cc:654] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.331938 6531 raft_consensus.cc:359] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.332307 6531 raft_consensus.cc:385] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.332448 6531 raft_consensus.cc:740] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.332881 6531 consensus_queue.cc:260] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.334830 6531 ts_tablet_manager.cc:1434] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 07:58:26.335314 6531 tablet_bootstrap.cc:492] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.337832 6531 tablet_bootstrap.cc:654] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.347260 6531 tablet_bootstrap.cc:492] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.347434 6531 ts_tablet_manager.cc:1403] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20260806 07:58:26.347998 6537 tablet_bootstrap.cc:492] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.348045 6532 tablet_bootstrap.cc:492] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.348166 6537 ts_tablet_manager.cc:1403] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.004s
I20260806 07:58:26.348206 6532 ts_tablet_manager.cc:1403] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.010s
I20260806 07:58:26.348400 6531 raft_consensus.cc:359] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.348578 6531 raft_consensus.cc:385] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.348675 6531 raft_consensus.cc:740] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.348949 6531 consensus_queue.cc:260] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.349026 6537 raft_consensus.cc:359] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.349195 6537 raft_consensus.cc:385] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.349093 6532 raft_consensus.cc:359] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.349278 6537 raft_consensus.cc:740] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.349290 6532 raft_consensus.cc:385] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.349392 6532 raft_consensus.cc:740] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.349500 6531 ts_tablet_manager.cc:1434] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:26.349570 6537 consensus_queue.cc:260] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.349758 6532 consensus_queue.cc:260] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.350132 6537 ts_tablet_manager.cc:1434] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:26.350718 6537 tablet_bootstrap.cc:492] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.351487 6531 tablet_bootstrap.cc:492] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.352458 6537 tablet_bootstrap.cc:654] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.353253 6531 tablet_bootstrap.cc:654] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.353801 6532 ts_tablet_manager.cc:1434] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260806 07:58:26.354116 6532 tablet_bootstrap.cc:492] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.355922 6532 tablet_bootstrap.cc:654] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:26.370857 6366 tablet.cc:2406] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 07:58:26.372506 6500 tablet.cc:2406] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:58:26.375438 6532 tablet_bootstrap.cc:492] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.375655 6532 ts_tablet_manager.cc:1403] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.010s
I20260806 07:58:26.376035 6537 tablet_bootstrap.cc:492] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.376216 6537 ts_tablet_manager.cc:1403] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.009s
I20260806 07:58:26.376317 6531 tablet_bootstrap.cc:492] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.376535 6531 ts_tablet_manager.cc:1403] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.006s
I20260806 07:58:26.376663 6532 raft_consensus.cc:359] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.376895 6532 raft_consensus.cc:385] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.376986 6532 raft_consensus.cc:740] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.377034 6537 raft_consensus.cc:359] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.377236 6537 raft_consensus.cc:385] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.377305 6537 raft_consensus.cc:740] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.377307 6532 consensus_queue.cc:260] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:26.377893 6228 tablet.cc:2406] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 07:58:26.377899 6531 raft_consensus.cc:359] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.378219 6531 raft_consensus.cc:385] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.378343 6531 raft_consensus.cc:740] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.377535 6537 consensus_queue.cc:260] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.379169 6537 ts_tablet_manager.cc:1434] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:26.378675 6531 consensus_queue.cc:260] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.382018 6531 ts_tablet_manager.cc:1434] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 07:58:26.382234 6546 raft_consensus.cc:493] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.382478 6531 tablet_bootstrap.cc:492] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.382465 6546 raft_consensus.cc:515] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.384260 6532 ts_tablet_manager.cc:1434] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260806 07:58:26.384689 6531 tablet_bootstrap.cc:654] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.385434 6546 leader_election.cc:290] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.388363 6532 tablet_bootstrap.cc:492] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.390960 6532 tablet_bootstrap.cc:654] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.400389 6532 tablet_bootstrap.cc:492] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.400596 6532 ts_tablet_manager.cc:1403] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260806 07:58:26.401769 6532 raft_consensus.cc:359] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.402040 6532 raft_consensus.cc:385] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.402133 6532 raft_consensus.cc:740] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.402563 6532 consensus_queue.cc:260] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.406621 6532 ts_tablet_manager.cc:1434] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260806 07:58:26.407058 6532 tablet_bootstrap.cc:492] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.408432 6544 raft_consensus.cc:493] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.408766 6544 raft_consensus.cc:515] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.412178 6537 tablet_bootstrap.cc:492] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.414709 6544 leader_election.cc:290] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.419984 6537 tablet_bootstrap.cc:654] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.432955 6532 tablet_bootstrap.cc:654] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.442117 6531 tablet_bootstrap.cc:492] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.442417 6531 ts_tablet_manager.cc:1403] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.060s user 0.035s sys 0.021s
I20260806 07:58:26.443624 6531 raft_consensus.cc:359] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.444012 6531 raft_consensus.cc:385] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.444221 6531 raft_consensus.cc:740] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.444727 6531 consensus_queue.cc:260] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.445426 6531 ts_tablet_manager.cc:1434] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:26.445942 6531 tablet_bootstrap.cc:492] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.448041 6531 tablet_bootstrap.cc:654] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.449649 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7d632b15694509ac4cce794498fbb3" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.450134 6180 raft_consensus.cc:2468] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.450913 6387 leader_election.cc:304] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.451462 6546 raft_consensus.cc:2804] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.451637 6546 raft_consensus.cc:493] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.451740 6546 raft_consensus.cc:3060] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.453964 6532 tablet_bootstrap.cc:492] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.454134 6532 ts_tablet_manager.cc:1403] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.000s
I20260806 07:58:26.454841 6546 raft_consensus.cc:515] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.454967 6532 raft_consensus.cc:359] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.455159 6532 raft_consensus.cc:385] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.455231 6532 raft_consensus.cc:740] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.455519 6537 tablet_bootstrap.cc:492] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.455657 6537 ts_tablet_manager.cc:1403] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.008s
I20260806 07:58:26.455575 6532 consensus_queue.cc:260] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.456069 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7d632b15694509ac4cce794498fbb3" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.456254 6180 raft_consensus.cc:3060] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.456452 6537 raft_consensus.cc:359] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.456633 6537 raft_consensus.cc:385] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.456727 6537 raft_consensus.cc:740] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.456943 6537 consensus_queue.cc:260] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.457896 6532 ts_tablet_manager.cc:1434] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:26.458235 6532 tablet_bootstrap.cc:492] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.459604 6180 raft_consensus.cc:2468] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.460484 6387 leader_election.cc:304] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.460950 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bce3ad6757940269b58768139f10bbf" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.461391 6546 leader_election.cc:290] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.461812 6546 raft_consensus.cc:2804] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.462368 6546 raft_consensus.cc:697] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:26.463172 6546 consensus_queue.cc:237] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.463725 6537 ts_tablet_manager.cc:1434] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
W20260806 07:58:26.465516 6113 leader_election.cc:343] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.466737 6537 tablet_bootstrap.cc:492] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.468562 6537 tablet_bootstrap.cc:654] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.470575 6532 tablet_bootstrap.cc:654] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.471621 6538 raft_consensus.cc:493] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.471856 6538 raft_consensus.cc:515] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.473254 6532 tablet_bootstrap.cc:492] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.473425 6532 ts_tablet_manager.cc:1403] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260806 07:58:26.473361 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bce3ad6757940269b58768139f10bbf" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
W20260806 07:58:26.474557 6114 leader_election.cc:343] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.474681 6532 raft_consensus.cc:359] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.474706 6114 leader_election.cc:304] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:26.474860 6532 raft_consensus.cc:385] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.474942 6532 raft_consensus.cc:740] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.475319 6532 consensus_queue.cc:260] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.476917 6537 tablet_bootstrap.cc:492] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.477036 6538 leader_election.cc:290] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.477131 6537 ts_tablet_manager.cc:1403] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260806 07:58:26.476929 6016 catalog_manager.cc:5697] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:26.477646 6544 raft_consensus.cc:2749] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:26.479808 6531 tablet_bootstrap.cc:492] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.480062 6531 ts_tablet_manager.cc:1403] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.000s
I20260806 07:58:26.481385 6531 raft_consensus.cc:359] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.481715 6531 raft_consensus.cc:385] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.481843 6531 raft_consensus.cc:740] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.482419 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7d632b15694509ac4cce794498fbb3" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.482385 6537 raft_consensus.cc:359] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.482666 6318 raft_consensus.cc:2468] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.482484 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7d632b15694509ac4cce794498fbb3" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.482661 6531 consensus_queue.cc:260] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.482887 6537 raft_consensus.cc:385] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.482848 6317 raft_consensus.cc:3060] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.483464 6531 ts_tablet_manager.cc:1434] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:26.482975 6537 raft_consensus.cc:740] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.484018 6537 consensus_queue.cc:260] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.484575 6537 ts_tablet_manager.cc:1434] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260806 07:58:26.485569 6531 tablet_bootstrap.cc:492] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.487545 6531 tablet_bootstrap.cc:654] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.489553 6532 ts_tablet_manager.cc:1434] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260806 07:58:26.489950 6532 tablet_bootstrap.cc:492] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.491837 6532 tablet_bootstrap.cc:654] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.492162 6317 raft_consensus.cc:2468] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.493134 6537 tablet_bootstrap.cc:492] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.495015 6537 tablet_bootstrap.cc:654] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.497025 6532 tablet_bootstrap.cc:492] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.497284 6532 ts_tablet_manager.cc:1403] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 07:58:26.498368 6532 raft_consensus.cc:359] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.498687 6532 raft_consensus.cc:385] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.498821 6532 raft_consensus.cc:740] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.499186 6532 consensus_queue.cc:260] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.499938 6532 ts_tablet_manager.cc:1434] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:26.500365 6532 tablet_bootstrap.cc:492] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.502689 6532 tablet_bootstrap.cc:654] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.524724 6572 raft_consensus.cc:493] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.524920 6572 raft_consensus.cc:515] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.526010 6572 leader_election.cc:290] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.534060 6537 tablet_bootstrap.cc:492] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.534175 6544 raft_consensus.cc:493] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.534265 6537 ts_tablet_manager.cc:1403] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.013s
I20260806 07:58:26.534518 6544 raft_consensus.cc:515] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.535184 6537 raft_consensus.cc:359] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.535310 6544 leader_election.cc:290] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.535377 6537 raft_consensus.cc:385] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.535481 6537 raft_consensus.cc:740] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.535729 6537 consensus_queue.cc:260] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.536283 6537 ts_tablet_manager.cc:1434] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:26.536623 6537 tablet_bootstrap.cc:492] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.538125 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727a2816995c4456b2210e7ad8913e3c" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.538403 6453 raft_consensus.cc:2468] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.538566 6537 tablet_bootstrap.cc:654] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.540946 6531 tablet_bootstrap.cc:492] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.541115 6531 ts_tablet_manager.cc:1403] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.019s
I20260806 07:58:26.541531 6114 leader_election.cc:304] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.542121 6531 raft_consensus.cc:359] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.542150 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727a2816995c4456b2210e7ad8913e3c" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.542317 6531 raft_consensus.cc:385] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.542407 6531 raft_consensus.cc:740] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.542402 6317 raft_consensus.cc:2468] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.542619 6583 raft_consensus.cc:493] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.542690 6531 consensus_queue.cc:260] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.542783 6583 raft_consensus.cc:515] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.543217 6544 raft_consensus.cc:2804] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.543354 6544 raft_consensus.cc:493] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.543452 6544 raft_consensus.cc:3060] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.543768 6583 leader_election.cc:290] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.545229 6531 ts_tablet_manager.cc:1434] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 07:58:26.547353 6531 tablet_bootstrap.cc:492] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.548166 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d04186bb02946928cc22152266d8a3f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.548985 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7d632b15694509ac4cce794498fbb3" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:26.549108 6114 leader_election.cc:343] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.549171 6531 tablet_bootstrap.cc:654] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.549408 6180 raft_consensus.cc:2393] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:26.549752 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d04186bb02946928cc22152266d8a3f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.549995 6317 raft_consensus.cc:2468] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.550048 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d6639b2db4f4cb99f33465ace8f0294" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.550382 6179 raft_consensus.cc:2468] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.550813 6537 tablet_bootstrap.cc:492] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.551317 6537 ts_tablet_manager.cc:1403] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
I20260806 07:58:26.551509 6113 leader_election.cc:304] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 5d4b117032604585af3544cb030a10f2
I20260806 07:58:26.551827 6583 raft_consensus.cc:2804] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.551962 6583 raft_consensus.cc:493] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.552069 6583 raft_consensus.cc:3060] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.552294 6537 raft_consensus.cc:359] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.552400 6252 leader_election.cc:304] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.552510 6537 raft_consensus.cc:385] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.552613 6537 raft_consensus.cc:740] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.552825 6572 raft_consensus.cc:2804] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.552932 6572 raft_consensus.cc:493] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.552896 6537 consensus_queue.cc:260] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.553014 6572 raft_consensus.cc:3060] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.553416 6537 ts_tablet_manager.cc:1434] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:26.553334 6544 raft_consensus.cc:515] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.553795 6537 tablet_bootstrap.cc:492] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.555801 6537 tablet_bootstrap.cc:654] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.555766 6583 raft_consensus.cc:515] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.556620 6544 leader_election.cc:290] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.557250 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727a2816995c4456b2210e7ad8913e3c" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.557469 6318 raft_consensus.cc:3060] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.557438 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727a2816995c4456b2210e7ad8913e3c" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.557690 6453 raft_consensus.cc:3060] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.559191 6568 raft_consensus.cc:493] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.559338 6568 raft_consensus.cc:515] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.560316 6568 leader_election.cc:290] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.560837 6318 raft_consensus.cc:2468] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.561362 6113 leader_election.cc:304] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.561946 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d04186bb02946928cc22152266d8a3f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.562151 6317 raft_consensus.cc:3060] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.562222 6572 raft_consensus.cc:515] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.562865 6572 leader_election.cc:290] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.564939 6317 raft_consensus.cc:2468] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.565374 6544 raft_consensus.cc:2804] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.565608 6544 raft_consensus.cc:697] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:26.565486 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d6639b2db4f4cb99f33465ace8f0294" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.565817 6179 raft_consensus.cc:3060] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.566836 6113 leader_election.cc:304] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.566293 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff476732379d406e8093e43da857d681" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.567378 6318 raft_consensus.cc:2468] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.567317 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7d632b15694509ac4cce794498fbb3" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.567894 6583 leader_election.cc:290] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.568264 6384 leader_election.cc:304] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:26.568557 6568 raft_consensus.cc:2804] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.568661 6568 raft_consensus.cc:493] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.568722 6583 raft_consensus.cc:2804] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.568756 6568 raft_consensus.cc:3060] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.568871 6583 raft_consensus.cc:697] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:26.569301 6583 consensus_queue.cc:237] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.571405 6179 raft_consensus.cc:2468] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.572024 6453 raft_consensus.cc:2468] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.572542 6568 raft_consensus.cc:515] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.573149 6544 consensus_queue.cc:237] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.574131 6568 leader_election.cc:290] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.574995 6572 raft_consensus.cc:493] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.575035 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d04186bb02946928cc22152266d8a3f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.575553 6531 tablet_bootstrap.cc:492] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
W20260806 07:58:26.575548 6114 leader_election.cc:343] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.575304 6572 raft_consensus.cc:515] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.575753 6531 ts_tablet_manager.cc:1403] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.000s
I20260806 07:58:26.576671 6531 raft_consensus.cc:359] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.576942 6531 raft_consensus.cc:385] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.577049 6531 raft_consensus.cc:740] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.577361 6531 consensus_queue.cc:260] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.577527 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d6639b2db4f4cb99f33465ace8f0294" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.577781 6452 raft_consensus.cc:3060] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.578084 6531 ts_tablet_manager.cc:1434] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 07:58:26.579774 6531 tablet_bootstrap.cc:492] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.579883 6587 raft_consensus.cc:493] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.580021 6587 raft_consensus.cc:515] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.580778 6587 leader_election.cc:290] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.581286 6252 leader_election.cc:304] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.581363 6452 raft_consensus.cc:2468] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.581327 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6305d69bfe42420a92ca7a9fc36ec2f6" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.581630 6538 raft_consensus.cc:2804] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.581671 6451 raft_consensus.cc:2468] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.581800 6531 tablet_bootstrap.cc:654] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.582206 6114 leader_election.cc:304] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.582577 6250 leader_election.cc:304] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:26.583236 6572 leader_election.cc:290] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.585434 6572 raft_consensus.cc:2749] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:26.585829 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff476732379d406e8093e43da857d681" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.586949 6317 raft_consensus.cc:3060] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.588104 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6305d69bfe42420a92ca7a9fc36ec2f6" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.588706 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d6639b2db4f4cb99f33465ace8f0294" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
W20260806 07:58:26.589192 6113 leader_election.cc:343] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.589217 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523ed5cfaa914101a9946b4391aad3c0" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.589669 6453 raft_consensus.cc:2376] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1. Re-sending same reply.
W20260806 07:58:26.590111 6252 leader_election.cc:343] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.590348 6532 tablet_bootstrap.cc:492] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.590528 6532 ts_tablet_manager.cc:1403] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.090s user 0.010s sys 0.023s
I20260806 07:58:26.591389 6532 raft_consensus.cc:359] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.591665 6532 raft_consensus.cc:385] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.591765 6532 raft_consensus.cc:740] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.592010 6532 consensus_queue.cc:260] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.592608 6532 ts_tablet_manager.cc:1434] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:26.593691 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff476732379d406e8093e43da857d681" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.593835 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff476732379d406e8093e43da857d681" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.592653 6581 raft_consensus.cc:2804] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.594931 6581 raft_consensus.cc:493] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.595155 6581 raft_consensus.cc:3060] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:58:26.595269 6387 leader_election.cc:343] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
W20260806 07:58:26.595721 6387 leader_election.cc:343] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.601388 6531 tablet_bootstrap.cc:492] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.601570 6531 ts_tablet_manager.cc:1403] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.003s
I20260806 07:58:26.602617 6531 raft_consensus.cc:359] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.602818 6531 raft_consensus.cc:385] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.602936 6531 raft_consensus.cc:740] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.603231 6531 consensus_queue.cc:260] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.603935 6531 ts_tablet_manager.cc:1434] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:26.604250 6531 tablet_bootstrap.cc:492] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.606405 6531 tablet_bootstrap.cc:654] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.610102 6017 catalog_manager.cc:5697] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.613752 6532 tablet_bootstrap.cc:492] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.614931 6317 raft_consensus.cc:2468] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.615406 6537 tablet_bootstrap.cc:492] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.615585 6537 ts_tablet_manager.cc:1403] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.030s
I20260806 07:58:26.615828 6384 leader_election.cc:304] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:26.616437 6568 raft_consensus.cc:2804] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.616466 6537 raft_consensus.cc:359] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.616590 6568 raft_consensus.cc:697] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:26.616659 6537 raft_consensus.cc:385] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.616752 6537 raft_consensus.cc:740] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.616910 6568 consensus_queue.cc:237] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.617122 6532 tablet_bootstrap.cc:654] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.617066 6537 consensus_queue.cc:260] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.617738 6537 ts_tablet_manager.cc:1434] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 07:58:26.618033 6537 tablet_bootstrap.cc:492] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.619810 6537 tablet_bootstrap.cc:654] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.619910 6581 raft_consensus.cc:515] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.621214 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6305d69bfe42420a92ca7a9fc36ec2f6" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.622079 6317 raft_consensus.cc:3060] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.627301 6317 raft_consensus.cc:2468] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.628407 6113 leader_election.cc:304] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.628774 6544 raft_consensus.cc:2804] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.628940 6544 raft_consensus.cc:697] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:26.629220 6544 consensus_queue.cc:237] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.629455 6531 tablet_bootstrap.cc:492] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.629659 6531 ts_tablet_manager.cc:1403] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.004s
I20260806 07:58:26.630570 6531 raft_consensus.cc:359] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.630864 6531 raft_consensus.cc:385] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.630967 6531 raft_consensus.cc:740] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.631268 6531 consensus_queue.cc:260] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.632072 6531 ts_tablet_manager.cc:1434] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:26.634239 6531 tablet_bootstrap.cc:492] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.637419 6581 leader_election.cc:290] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.637688 6531 tablet_bootstrap.cc:654] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.638775 6532 tablet_bootstrap.cc:492] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.638985 6532 ts_tablet_manager.cc:1403] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.000s
I20260806 07:58:26.639927 6531 tablet_bootstrap.cc:492] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.639997 6532 raft_consensus.cc:359] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.640130 6531 ts_tablet_manager.cc:1403] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 07:58:26.640230 6532 raft_consensus.cc:385] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.640314 6532 raft_consensus.cc:740] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.640635 6532 consensus_queue.cc:260] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.641184 6531 raft_consensus.cc:359] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.641356 6532 ts_tablet_manager.cc:1434] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:26.641480 6531 raft_consensus.cc:385] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.641665 6531 raft_consensus.cc:740] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.641726 6532 tablet_bootstrap.cc:492] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.642014 6531 consensus_queue.cc:260] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.642724 6531 ts_tablet_manager.cc:1434] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 07:58:26.643105 6531 tablet_bootstrap.cc:492] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.643936 6532 tablet_bootstrap.cc:654] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.645210 6531 tablet_bootstrap.cc:654] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.645849 6537 tablet_bootstrap.cc:492] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.646266 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6305d69bfe42420a92ca7a9fc36ec2f6" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.646556 6453 raft_consensus.cc:3060] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.646903 6537 ts_tablet_manager.cc:1403] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.005s
I20260806 07:58:26.648444 6537 raft_consensus.cc:359] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.648762 6537 raft_consensus.cc:385] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.648862 6537 raft_consensus.cc:740] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.651659 6537 consensus_queue.cc:260] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.652307 6453 raft_consensus.cc:2468] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.652596 6537 ts_tablet_manager.cc:1434] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260806 07:58:26.653031 6537 tablet_bootstrap.cc:492] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.655452 6537 tablet_bootstrap.cc:654] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:26.656075 6250 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.202.195:36665, user_credentials={real_user=slave}} blocked reactor thread for 79214.9us
I20260806 07:58:26.657372 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523ed5cfaa914101a9946b4391aad3c0" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.657655 6451 raft_consensus.cc:2468] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.660159 6531 tablet_bootstrap.cc:492] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.660357 6531 ts_tablet_manager.cc:1403] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.012s
I20260806 07:58:26.661383 6531 raft_consensus.cc:359] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.661604 6544 raft_consensus.cc:493] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.661659 6531 raft_consensus.cc:385] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.661738 6531 raft_consensus.cc:740] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.661792 6544 raft_consensus.cc:515] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.662024 6531 consensus_queue.cc:260] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.662671 6531 ts_tablet_manager.cc:1434] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 07:58:26.663074 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d0ae6d095640eea8caa5853c1fbb2a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.663304 6317 raft_consensus.cc:2468] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.663563 6544 leader_election.cc:290] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.663651 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d0ae6d095640eea8caa5853c1fbb2a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.663843 6113 leader_election.cc:304] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.663883 6453 raft_consensus.cc:2468] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.664208 6544 raft_consensus.cc:2804] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.664399 6544 raft_consensus.cc:493] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.664582 6544 raft_consensus.cc:3060] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.664280 6537 tablet_bootstrap.cc:492] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.664934 6537 ts_tablet_manager.cc:1403] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260806 07:58:26.665730 6592 raft_consensus.cc:493] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.665797 6531 tablet_bootstrap.cc:492] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.665925 6592 raft_consensus.cc:515] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.666985 6592 leader_election.cc:290] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.667559 6531 tablet_bootstrap.cc:654] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.667819 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d800927238145aea3e3c82d1bf158e1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.667955 6544 raft_consensus.cc:515] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.668095 6179 raft_consensus.cc:2468] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.668655 6544 leader_election.cc:290] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.668900 6252 leader_election.cc:304] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.669739 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d0ae6d095640eea8caa5853c1fbb2a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.669971 6317 raft_consensus.cc:3060] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.670230 6592 raft_consensus.cc:2804] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.670374 6592 raft_consensus.cc:493] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.670506 6592 raft_consensus.cc:3060] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.671797 6537 raft_consensus.cc:359] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.671996 6537 raft_consensus.cc:385] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.672079 6537 raft_consensus.cc:740] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.672302 6537 consensus_queue.cc:260] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.672820 6537 ts_tablet_manager.cc:1434] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260806 07:58:26.673122 6537 tablet_bootstrap.cc:492] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.673388 6317 raft_consensus.cc:2468] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.674188 6113 leader_election.cc:304] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.674608 6581 raft_consensus.cc:2804] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.674780 6581 raft_consensus.cc:697] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:26.674716 6592 raft_consensus.cc:515] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.675076 6581 consensus_queue.cc:237] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.676121 6592 leader_election.cc:290] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.676252 6531 tablet_bootstrap.cc:492] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.676407 6531 ts_tablet_manager.cc:1403] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260806 07:58:26.676666 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d0ae6d095640eea8caa5853c1fbb2a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.676862 6453 raft_consensus.cc:3060] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.677277 6531 raft_consensus.cc:359] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.677335 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d800927238145aea3e3c82d1bf158e1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.677457 6531 raft_consensus.cc:385] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.677531 6179 raft_consensus.cc:3060] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.677567 6531 raft_consensus.cc:740] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.682238 6179 raft_consensus.cc:2468] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.682564 6546 raft_consensus.cc:493] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.682780 6546 raft_consensus.cc:515] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.683079 6250 leader_election.cc:304] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:26.683363 6592 raft_consensus.cc:2804] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.683466 6592 raft_consensus.cc:493] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.683586 6546 leader_election.cc:290] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.683686 6592 raft_consensus.cc:3060] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.681495 6531 consensus_queue.cc:260] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.684324 6531 ts_tablet_manager.cc:1434] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 07:58:26.684636 6531 tablet_bootstrap.cc:492] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.684931 6252 leader_election.cc:304] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.685436 6572 raft_consensus.cc:2804] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.685711 6572 raft_consensus.cc:697] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:26.686386 6572 consensus_queue.cc:237] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.686815 6592 raft_consensus.cc:515] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.687081 6531 tablet_bootstrap.cc:654] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.687551 6592 leader_election.cc:290] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.688177 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523ed5cfaa914101a9946b4391aad3c0" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.688405 6179 raft_consensus.cc:3060] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.692371 6179 raft_consensus.cc:2468] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.693049 6252 leader_election.cc:304] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.694254 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcae8fb644044d99a96dd9fb716965c" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.694617 6317 raft_consensus.cc:2468] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.695251 6384 leader_election.cc:304] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:26.695775 6546 raft_consensus.cc:2804] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.695982 6546 raft_consensus.cc:493] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.696115 6546 raft_consensus.cc:3060] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.699427 6546 raft_consensus.cc:515] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.700819 6546 leader_election.cc:290] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.700822 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcae8fb644044d99a96dd9fb716965c" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.701110 6318 raft_consensus.cc:3060] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.701758 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcae8fb644044d99a96dd9fb716965c" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
W20260806 07:58:26.702708 6387 leader_election.cc:343] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.705260 6318 raft_consensus.cc:2468] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.705762 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcae8fb644044d99a96dd9fb716965c" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.706210 6384 leader_election.cc:304] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
W20260806 07:58:26.706663 6387 leader_election.cc:343] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:26.706938 6546 raft_consensus.cc:2804] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.707145 6546 raft_consensus.cc:697] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:26.707471 6546 consensus_queue.cc:237] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.709707 6572 raft_consensus.cc:2804] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.709986 6572 raft_consensus.cc:697] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:26.710537 6572 consensus_queue.cc:237] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.710780 6538 raft_consensus.cc:697] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:26.711170 6538 consensus_queue.cc:237] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.713209 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d800927238145aea3e3c82d1bf158e1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.713413 6451 raft_consensus.cc:2468] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.675097 6537 tablet_bootstrap.cc:654] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.713768 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d800927238145aea3e3c82d1bf158e1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.713958 6452 raft_consensus.cc:3060] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.716575 6532 tablet_bootstrap.cc:492] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.716730 6532 ts_tablet_manager.cc:1403] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.075s user 0.021s sys 0.024s
I20260806 07:58:26.717490 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "523ed5cfaa914101a9946b4391aad3c0" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.717698 6451 raft_consensus.cc:3060] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.717500 6532 raft_consensus.cc:359] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.718158 6532 raft_consensus.cc:385] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.718250 6532 raft_consensus.cc:740] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.718505 6532 consensus_queue.cc:260] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.718655 6537 tablet_bootstrap.cc:492] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.718658 6452 raft_consensus.cc:2468] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.718811 6537 ts_tablet_manager.cc:1403] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.046s user 0.004s sys 0.000s
I20260806 07:58:26.718976 6453 raft_consensus.cc:2468] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.719604 6532 ts_tablet_manager.cc:1434] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:26.719651 6537 raft_consensus.cc:359] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.719839 6537 raft_consensus.cc:385] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.719901 6532 tablet_bootstrap.cc:492] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.719911 6537 raft_consensus.cc:740] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.720187 6537 consensus_queue.cc:260] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.720710 6537 ts_tablet_manager.cc:1434] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:26.721024 6537 tablet_bootstrap.cc:492] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.721750 6532 tablet_bootstrap.cc:654] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.722465 6451 raft_consensus.cc:2468] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.722971 6537 tablet_bootstrap.cc:654] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.726787 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.727172 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.727504 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.728235 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.728549 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.728821 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.729233 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.732054 6531 tablet_bootstrap.cc:492] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.732240 6531 ts_tablet_manager.cc:1403] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.008s
I20260806 07:58:26.732583 6572 raft_consensus.cc:493] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.732753 6572 raft_consensus.cc:515] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.733106 6531 raft_consensus.cc:359] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.733377 6531 raft_consensus.cc:385] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.733461 6531 raft_consensus.cc:740] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.733570 6572 leader_election.cc:290] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.734390 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bce3ad6757940269b58768139f10bbf" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.734661 6179 raft_consensus.cc:2468] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.735193 6252 leader_election.cc:304] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.735514 6572 raft_consensus.cc:2804] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.735634 6572 raft_consensus.cc:493] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.735773 6572 raft_consensus.cc:3060] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.737030 6014 catalog_manager.cc:5697] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.738451 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.738847 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.739149 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.739459 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.741667 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.741983 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.742296 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.744635 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.745020 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.745306 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.745494 6531 consensus_queue.cc:260] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.746676 6531 ts_tablet_manager.cc:1434] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260806 07:58:26.747020 6531 tablet_bootstrap.cc:492] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:26.749009 6531 tablet_bootstrap.cc:654] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.750497 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":174,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:26.751381 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bce3ad6757940269b58768139f10bbf" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.751797 6451 raft_consensus.cc:2468] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.752182 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.000s Metrics: {"bytes_written":471,"delete_count":0,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"mutex_wait_us":2119,"reinsert_count":0,"update_count":2}
I20260806 07:58:26.752473 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":6722}
I20260806 07:58:26.752533 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:26.752903 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.129197
I20260806 07:58:26.753183 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.129197
I20260806 07:58:26.753482 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.129197
I20260806 07:58:26.753677 6572 raft_consensus.cc:515] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.754143 6537 tablet_bootstrap.cc:492] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.754298 6537 ts_tablet_manager.cc:1403] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.011s
I20260806 07:58:26.754432 6572 leader_election.cc:290] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.755152 6537 raft_consensus.cc:359] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.755323 6537 raft_consensus.cc:385] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.755417 6537 raft_consensus.cc:740] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.756083 6532 tablet_bootstrap.cc:492] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.756009 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bce3ad6757940269b58768139f10bbf" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.756189 6179 raft_consensus.cc:3060] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.756238 6532 ts_tablet_manager.cc:1403] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.010s
I20260806 07:58:26.752310 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":6680}
I20260806 07:58:26.756953 6532 raft_consensus.cc:359] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.757149 6532 raft_consensus.cc:385] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.757464 6532 raft_consensus.cc:740] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.757792 6537 consensus_queue.cc:260] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.758173 6532 consensus_queue.cc:260] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.758862 6537 ts_tablet_manager.cc:1434] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:26.759176 6537 tablet_bootstrap.cc:492] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.759243 6179 raft_consensus.cc:2468] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.760957 6537 tablet_bootstrap.cc:654] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.761444 6252 leader_election.cc:304] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.762285 6532 ts_tablet_manager.cc:1434] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 07:58:26.762415 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.762598 6532 tablet_bootstrap.cc:492] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:26.762724 6572 raft_consensus.cc:2804] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.762729 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.762857 6572 raft_consensus.cc:697] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:26.763114 6572 consensus_queue.cc:237] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.764689 6532 tablet_bootstrap.cc:654] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.764635 6546 raft_consensus.cc:493] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.764811 6546 raft_consensus.cc:515] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.765758 6546 leader_election.cc:290] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.766554 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71ed2aa2bf43ffbc98dda21fc73eac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.766788 6318 raft_consensus.cc:2468] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.767338 6384 leader_election.cc:304] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:26.767879 6546 raft_consensus.cc:2804] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.768009 6546 raft_consensus.cc:493] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.768133 6546 raft_consensus.cc:3060] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.768172 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71ed2aa2bf43ffbc98dda21fc73eac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.768391 6179 raft_consensus.cc:2468] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.771857 6546 raft_consensus.cc:515] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.773048 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71ed2aa2bf43ffbc98dda21fc73eac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.773304 6179 raft_consensus.cc:3060] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.776409 6531 tablet_bootstrap.cc:492] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:26.776585 6531 ts_tablet_manager.cc:1403] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.020s
I20260806 07:58:26.777001 6179 raft_consensus.cc:2468] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.777575 6531 raft_consensus.cc:359] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.777714 6387 leader_election.cc:304] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.777810 6531 raft_consensus.cc:385] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.778061 6568 raft_consensus.cc:2804] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.778169 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bce3ad6757940269b58768139f10bbf" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.778441 6451 raft_consensus.cc:3060] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.780014 6016 catalog_manager.cc:5697] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:26.781144 6016 catalog_manager.cc:5697] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:26.782392 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.782758 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.783052 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.783406 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.783530 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d71ed2aa2bf43ffbc98dda21fc73eac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.783783 6318 raft_consensus.cc:3060] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.784698 6532 tablet_bootstrap.cc:492] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:26.784861 6532 ts_tablet_manager.cc:1403] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.002s
I20260806 07:58:26.785420 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.785727 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.777922 6531 raft_consensus.cc:740] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.785784 6532 raft_consensus.cc:359] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.785950 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.785995 6532 raft_consensus.cc:385] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.786082 6532 raft_consensus.cc:740] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.786262 6531 consensus_queue.cc:260] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.786340 6532 consensus_queue.cc:260] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.786580 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.311659
I20260806 07:58:26.786917 6318 raft_consensus.cc:2468] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.787014 6532 ts_tablet_manager.cc:1434] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:26.786985 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.311659
I20260806 07:58:26.787400 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.311659
I20260806 07:58:26.787909 6546 leader_election.cc:290] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.788269 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.788476 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.788853 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:26.789240 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.789913 6546 raft_consensus.cc:493] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.790159 6546 raft_consensus.cc:515] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.792924 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59dfcb1b67340e2b93fc9e8644df607" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.793206 6179 raft_consensus.cc:2468] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.793817 6387 leader_election.cc:304] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.795454 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59dfcb1b67340e2b93fc9e8644df607" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.795778 6318 raft_consensus.cc:2468] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.795913 6622 raft_consensus.cc:2804] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.796010 6622 raft_consensus.cc:493] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.796118 6622 raft_consensus.cc:3060] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.797618 6451 raft_consensus.cc:2468] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.798364 6546 leader_election.cc:290] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.798430 6531 ts_tablet_manager.cc:1434] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.022s user 0.002s sys 0.003s
I20260806 07:58:26.798699 6568 raft_consensus.cc:697] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:26.798997 6568 consensus_queue.cc:237] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.806730 6622 raft_consensus.cc:515] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.808050 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59dfcb1b67340e2b93fc9e8644df607" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.805400 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.012s sys 0.004s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6821,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":12193,"dirs.run_cpu_time_us":342,"dirs.run_wall_time_us":3449,"lbm_read_time_us":6152,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":26,"lbm_write_time_us":2278,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":13978,"threads_started":2,"update_count":3}
I20260806 07:58:26.808323 6179 raft_consensus.cc:3060] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.808992 6537 tablet_bootstrap.cc:492] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.809185 6537 ts_tablet_manager.cc:1403] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.050s user 0.016s sys 0.008s
I20260806 07:58:26.810102 6537 raft_consensus.cc:359] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.810326 6537 raft_consensus.cc:385] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.810415 6537 raft_consensus.cc:740] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.810667 6537 consensus_queue.cc:260] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.811317 6537 ts_tablet_manager.cc:1434] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 07:58:26.811657 6537 tablet_bootstrap.cc:492] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:26.813820 6537 tablet_bootstrap.cc:654] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:26.816864 6179 raft_consensus.cc:2468] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.817582 6387 leader_election.cc:304] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.818080 6546 raft_consensus.cc:2804] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.818310 6546 raft_consensus.cc:697] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:26.818715 6546 consensus_queue.cc:237] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.819805 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.820200 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.820501 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.820791 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.821228 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.821635 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.821892 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.823101 6537 tablet_bootstrap.cc:492] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:26.823251 6537 ts_tablet_manager.cc:1403] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260806 07:58:26.824337 6572 raft_consensus.cc:493] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.824522 6572 raft_consensus.cc:515] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.824647 6583 raft_consensus.cc:493] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.824784 6583 raft_consensus.cc:515] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.825270 6572 leader_election.cc:290] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.825435 6583 leader_election.cc:290] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.824108 6537 raft_consensus.cc:359] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.825819 6537 raft_consensus.cc:385] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:26.825927 6537 raft_consensus.cc:740] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:26.826285 6537 consensus_queue.cc:260] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.826459 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f7851b0e26d40e48bd60a7671b643b9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.826617 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.826674 6318 raft_consensus.cc:2468] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:26.826851 6537 ts_tablet_manager.cc:1434] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:26.827000 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.827236 6113 leader_election.cc:304] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.827626 6581 raft_consensus.cc:2804] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.827728 6581 raft_consensus.cc:493] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.827840 6581 raft_consensus.cc:3060] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.828990 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.005s sys 0.000s Metrics: {"bytes_written":471,"delete_count":0,"lbm_write_time_us":188,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:26.829737 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.830161 6581 raft_consensus.cc:515] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.830818 6581 leader_election.cc:290] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.832057 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7a45d28865477bbe3cef826d477a93" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.832283 6451 raft_consensus.cc:2468] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.832945 6014 catalog_manager.cc:5697] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.833638 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f7851b0e26d40e48bd60a7671b643b9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.833863 6318 raft_consensus.cc:3060] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.833793 6014 catalog_manager.cc:5697] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.834422 6014 catalog_manager.cc:5697] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.836735 6318 raft_consensus.cc:2468] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.837855 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f7851b0e26d40e48bd60a7671b643b9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.837879 6250 leader_election.cc:304] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:26.838058 6453 raft_consensus.cc:3060] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.838303 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f7851b0e26d40e48bd60a7671b643b9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:26.838790 6113 leader_election.cc:304] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.839108 6583 raft_consensus.cc:2804] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.839227 6583 raft_consensus.cc:697] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:26.839514 6583 consensus_queue.cc:237] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.841375 6453 raft_consensus.cc:2468] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:26.842681 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59dfcb1b67340e2b93fc9e8644df607" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.842880 6318 raft_consensus.cc:3060] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.843343 6572 raft_consensus.cc:2804] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.843508 6572 raft_consensus.cc:493] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.843668 6572 raft_consensus.cc:3060] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.847656 6318 raft_consensus.cc:2468] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.848819 6622 leader_election.cc:290] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.852932 6568 raft_consensus.cc:493] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.853094 6568 raft_consensus.cc:515] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.853348 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.016s sys 0.006s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6821,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":12540,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":983,"lbm_read_time_us":521,"lbm_reads_lt_1ms":27,"lbm_write_time_us":387,"lbm_writes_lt_1ms":9,"mutex_wait_us":41215,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":10798,"threads_started":2,"update_count":3}
I20260806 07:58:26.853803 6568 leader_election.cc:290] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079)
I20260806 07:58:26.854460 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6047b73f65e94f58b985a178aec1e568" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.854709 6318 raft_consensus.cc:2468] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.855188 6384 leader_election.cc:304] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:26.855491 6568 raft_consensus.cc:2804] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:26.855633 6568 raft_consensus.cc:493] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:26.855751 6568 raft_consensus.cc:3060] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.856267 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.005s sys 0.000s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":208,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:26.857652 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7a45d28865477bbe3cef826d477a93" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.857877 6179 raft_consensus.cc:2468] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:26.858783 6568 raft_consensus.cc:515] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.858794 6572 raft_consensus.cc:515] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.859743 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7a45d28865477bbe3cef826d477a93" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.859778 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6047b73f65e94f58b985a178aec1e568" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:26.859956 6180 raft_consensus.cc:3060] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.859985 6318 raft_consensus.cc:3060] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.862761 6318 raft_consensus.cc:2468] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.863224 6384 leader_election.cc:304] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:26.863688 6568 leader_election.cc:290] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079)
I20260806 07:58:26.863911 6568 raft_consensus.cc:2804] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.863864 6016 catalog_manager.cc:5697] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:26.864306 6568 raft_consensus.cc:697] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:26.864391 6016 catalog_manager.cc:5697] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:26.864578 6568 consensus_queue.cc:237] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:26.866035 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6047b73f65e94f58b985a178aec1e568" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.866264 6179 raft_consensus.cc:2468] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:26.866207 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6047b73f65e94f58b985a178aec1e568" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:26.866484 6178 raft_consensus.cc:3060] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.868212 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.868542 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.868799 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.869015 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.869894 6180 raft_consensus.cc:2468] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.870920 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.871230 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.871482 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.872148 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.309344
I20260806 07:58:26.872464 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.309344
I20260806 07:58:26.872524 6572 leader_election.cc:290] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:26.872742 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.309344
I20260806 07:58:26.873317 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.873570 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.874039 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:26.874019 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.005s sys 0.000s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":136,"lbm_writes_lt_1ms":4,"mutex_wait_us":24583,"reinsert_count":0,"update_count":2}
I20260806 07:58:26.874511 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.309344
I20260806 07:58:26.874784 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.309344
I20260806 07:58:26.875149 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7a45d28865477bbe3cef826d477a93" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:26.875362 6453 raft_consensus.cc:3060] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.878433 6453 raft_consensus.cc:2468] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:26.879144 6252 leader_election.cc:304] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:26.879053 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.004s sys 0.000s Metrics: {"bytes_written":474,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260806 07:58:26.879423 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.879459 6572 raft_consensus.cc:2804] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:26.879616 6572 raft_consensus.cc:697] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:26.879896 6572 consensus_queue.cc:237] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.880277 6178 raft_consensus.cc:2468] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:26.880247 6014 catalog_manager.cc:5697] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.880784 6014 catalog_manager.cc:5697] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.881271 6014 catalog_manager.cc:5697] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.882750 6014 catalog_manager.cc:5697] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.890946 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.891280 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.891498 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.897905 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.070s user 0.005s sys 0.000s Metrics: {"bytes_written":474,"delete_count":0,"lbm_write_time_us":136,"lbm_writes_lt_1ms":4,"mutex_wait_us":33965,"reinsert_count":0,"update_count":3}
I20260806 07:58:26.902002 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.902637 6016 catalog_manager.cc:5697] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.907204 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.077s user 0.002s sys 0.002s Metrics: {"bytes_written":547,"delete_count":0,"lbm_write_time_us":210,"lbm_writes_lt_1ms":4,"mutex_wait_us":67815,"reinsert_count":0,"update_count":6}
I20260806 07:58:26.911512 6014 catalog_manager.cc:5697] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:26.918123 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.918409 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.918684 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:26.921682 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.049s user 0.008s sys 0.004s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6953,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1597,"dirs.run_cpu_time_us":158,"dirs.run_wall_time_us":746,"lbm_read_time_us":620,"lbm_reads_lt_1ms":27,"lbm_write_time_us":320,"lbm_writes_lt_1ms":9,"mutex_wait_us":9304,"peak_mem_usage":32768,"reinsert_count":0,"update_count":5}
I20260806 07:58:26.925845 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.034s user 0.000s sys 0.005s Metrics: {"bytes_written":607,"delete_count":0,"lbm_write_time_us":139,"lbm_writes_lt_1ms":4,"mutex_wait_us":28515,"reinsert_count":0,"spinlock_wait_cycles":896,"update_count":7}
I20260806 07:58:26.930150 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.186268
I20260806 07:58:26.930501 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.186268
I20260806 07:58:26.930706 6568 raft_consensus.cc:493] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:26.930835 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":38800}
I20260806 07:58:26.930899 6568 raft_consensus.cc:515] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:26.931424 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.004s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":127,"lbm_writes_lt_1ms":4,"mutex_wait_us":33926,"reinsert_count":0,"update_count":1}
I20260806 07:58:26.931690 6568 leader_election.cc:290] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:26.932267 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:26.932379 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d04186bb02946928cc22152266d8a3f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:26.932987 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d04186bb02946928cc22152266d8a3f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:26.933353 6318 raft_consensus.cc:2393] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d4b117032604585af3544cb030a10f2 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:26.934516 6384 leader_election.cc:304] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:26.935105 6568 raft_consensus.cc:3060] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:26.938217 6568 raft_consensus.cc:2749] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:26.941330 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":22105}
I20260806 07:58:26.941725 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":22404}
I20260806 07:58:26.941560 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.044s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":7399,"cfile_init":6,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":86,"dirs.run_cpu_time_us":123,"dirs.run_wall_time_us":1665,"lbm_read_time_us":549,"lbm_reads_lt_1ms":33,"lbm_write_time_us":288,"lbm_writes_lt_1ms":9,"mutex_wait_us":9778,"peak_mem_usage":32768,"reinsert_count":0,"update_count":11}
I20260806 07:58:26.941253 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.002s sys 0.000s Metrics: {"bytes_written":494,"delete_count":0,"lbm_write_time_us":130,"lbm_writes_lt_1ms":4,"mutex_wait_us":17391,"reinsert_count":0,"update_count":2}
I20260806 07:58:26.942278 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.008s sys 0.002s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":7253,"cfile_init":6,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":94,"dirs.run_cpu_time_us":120,"dirs.run_wall_time_us":824,"lbm_read_time_us":575,"lbm_reads_lt_1ms":31,"lbm_write_time_us":321,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":8}
I20260806 07:58:26.948553 6568 consensus_queue.cc:1048] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:26.957422 6568 consensus_queue.cc:1048] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:26.965816 6581 consensus_queue.cc:1048] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:26.972031 6581 consensus_queue.cc:1048] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:26.982897 6581 consensus_queue.cc:1048] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:26.988436 6581 consensus_queue.cc:1048] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.046487 6568 consensus_queue.cc:1048] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.051066 6178 raft_consensus.cc:3060] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.055260 6546 consensus_queue.cc:1048] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.071952 6014 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-0" } modify_external_catalogs: true
I20260806 07:58:27.072228 6014 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-0" } modify_external_catalogs: true
I20260806 07:58:27.083819 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 6047b73f65e94f58b985a178aec1e568
I20260806 07:58:27.085287 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 6047b73f65e94f58b985a178aec1e568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.085378 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 6047b73f65e94f58b985a178aec1e568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.085860 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 6047b73f65e94f58b985a178aec1e568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.086902 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.087472 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.087818 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.088112 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.088382 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.088837 6648 tablet_replica.cc:333] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.089325 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.089532 6648 raft_consensus.cc:2243] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.089665 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.089891 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.089959 6648 raft_consensus.cc:2272] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.091559 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.091885 6648 ts_tablet_manager.cc:1916] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.092135 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.092610 6572 consensus_queue.cc:1048] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.093086 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 6305d69bfe42420a92ca7a9fc36ec2f6
I20260806 07:58:27.094288 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 6305d69bfe42420a92ca7a9fc36ec2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.094354 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.094698 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.095003 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.095227 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.095532 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.095956 6296 tablet_service.cc:1558] Processing DeleteTablet for tablet 6305d69bfe42420a92ca7a9fc36ec2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.096479 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 6305d69bfe42420a92ca7a9fc36ec2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.096774 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 1f7851b0e26d40e48bd60a7671b643b9
I20260806 07:58:27.097638 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f7851b0e26d40e48bd60a7671b643b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.098587 6296 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f7851b0e26d40e48bd60a7671b643b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.098982 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet f59dfcb1b67340e2b93fc9e8644df607
I20260806 07:58:27.099092 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f7851b0e26d40e48bd60a7671b643b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.099635 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet f59dfcb1b67340e2b93fc9e8644df607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.100714 6648 ts_tablet_manager.cc:1929] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.100770 6296 tablet_service.cc:1558] Processing DeleteTablet for tablet f59dfcb1b67340e2b93fc9e8644df607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.100929 6648 log.cc:1199] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/6047b73f65e94f58b985a178aec1e568
I20260806 07:58:27.101450 6648 ts_tablet_manager.cc:1950] T 6047b73f65e94f58b985a178aec1e568 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.101758 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet f59dfcb1b67340e2b93fc9e8644df607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.102191 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 4d7d632b15694509ac4cce794498fbb3
I20260806 07:58:27.103029 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d7d632b15694509ac4cce794498fbb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.103858 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 6047b73f65e94f58b985a178aec1e568 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.104363 6648 tablet_replica.cc:333] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.104367 6296 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d7d632b15694509ac4cce794498fbb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.104653 6648 raft_consensus.cc:2243] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.105039 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d7d632b15694509ac4cce794498fbb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.105104 6648 pending_rounds.cc:70] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Trying to abort 1 pending ops.
I20260806 07:58:27.105300 6648 pending_rounds.cc:77] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468724756000768 op_type: NO_OP noop_request { }
I20260806 07:58:27.105451 6648 raft_consensus.cc:2889] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:27.105396 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 1d800927238145aea3e3c82d1bf158e1
I20260806 07:58:27.105650 6648 raft_consensus.cc:2272] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.106101 6647 tablet_replica.cc:333] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.106252 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.106568 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d800927238145aea3e3c82d1bf158e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.106716 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d800927238145aea3e3c82d1bf158e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.106706 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.106812 6647 raft_consensus.cc:2243] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.107139 6647 raft_consensus.cc:2272] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.107658 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d800927238145aea3e3c82d1bf158e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.107472 6648 ts_tablet_manager.cc:1916] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.108157 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 727a2816995c4456b2210e7ad8913e3c
I20260806 07:58:27.108862 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 727a2816995c4456b2210e7ad8913e3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.108902 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 727a2816995c4456b2210e7ad8913e3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.110069 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 727a2816995c4456b2210e7ad8913e3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.110553 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 8d6639b2db4f4cb99f33465ace8f0294
I20260806 07:58:27.110862 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.006s sys 0.000s Metrics: {"bytes_written":993,"delete_count":0,"lbm_write_time_us":182,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:27.110906 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.004s sys 0.000s Metrics: {"bytes_written":993,"delete_count":0,"lbm_write_time_us":193,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:27.111459 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.129787
I20260806 07:58:27.111501 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d6639b2db4f4cb99f33465ace8f0294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.111856 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.129787
I20260806 07:58:27.112215 6649 tablet_replica.cc:333] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.112674 6649 raft_consensus.cc:2243] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.113217 6649 pending_rounds.cc:70] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Trying to abort 1 pending ops.
I20260806 07:58:27.113353 6649 pending_rounds.cc:77] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468725720096768 op_type: NO_OP noop_request { }
I20260806 07:58:27.113502 6649 raft_consensus.cc:2889] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:27.113646 6649 raft_consensus.cc:2272] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.110287 6647 ts_tablet_manager.cc:1916] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.111594 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d6639b2db4f4cb99f33465ace8f0294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.114769 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d6639b2db4f4cb99f33465ace8f0294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.114959 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet ff476732379d406e8093e43da857d681
I20260806 07:58:27.115779 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet ff476732379d406e8093e43da857d681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.115832 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet ff476732379d406e8093e43da857d681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.116509 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet ff476732379d406e8093e43da857d681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.116714 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 0d04186bb02946928cc22152266d8a3f
I20260806 07:58:27.117333 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d04186bb02946928cc22152266d8a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.117446 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d04186bb02946928cc22152266d8a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.118206 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d04186bb02946928cc22152266d8a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.118415 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 8d71ed2aa2bf43ffbc98dda21fc73eac
I20260806 07:58:27.118361 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.119081 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.119115 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.119760 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d71ed2aa2bf43ffbc98dda21fc73eac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.119958 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet fb7a45d28865477bbe3cef826d477a93
I20260806 07:58:27.120669 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet fb7a45d28865477bbe3cef826d477a93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.120689 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet fb7a45d28865477bbe3cef826d477a93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.121073 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 523ed5cfaa914101a9946b4391aad3c0
I20260806 07:58:27.121753 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 523ed5cfaa914101a9946b4391aad3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.121845 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 523ed5cfaa914101a9946b4391aad3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.123338 6649 ts_tablet_manager.cc:1916] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.124253 6648 ts_tablet_manager.cc:1929] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.124377 6648 log.cc:1199] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/6305d69bfe42420a92ca7a9fc36ec2f6
I20260806 07:58:27.124814 6648 ts_tablet_manager.cc:1950] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.125352 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet fb7a45d28865477bbe3cef826d477a93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.125563 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 523ed5cfaa914101a9946b4391aad3c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.125828 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 1bce3ad6757940269b58768139f10bbf
I20260806 07:58:27.125962 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 6305d69bfe42420a92ca7a9fc36ec2f6 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.126531 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bce3ad6757940269b58768139f10bbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.126617 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bce3ad6757940269b58768139f10bbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.127302 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bce3ad6757940269b58768139f10bbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.127828 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet cbcae8fb644044d99a96dd9fb716965c
I20260806 07:58:27.127626 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.011s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":7595,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":228,"lbm_read_time_us":585,"lbm_reads_lt_1ms":25,"lbm_write_time_us":344,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":17}
I20260806 07:58:27.128789 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet cbcae8fb644044d99a96dd9fb716965c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.129087 6590 consensus_queue.cc:1048] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.129359 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet cbcae8fb644044d99a96dd9fb716965c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.129670 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet cbcae8fb644044d99a96dd9fb716965c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.129886 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet b6d0ae6d095640eea8caa5853c1fbb2a
I20260806 07:58:27.130494 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet b6d0ae6d095640eea8caa5853c1fbb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:27.130607 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet b6d0ae6d095640eea8caa5853c1fbb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:27.133312 6647 ts_tablet_manager.cc:1929] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.133476 6647 log.cc:1199] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/6047b73f65e94f58b985a178aec1e568
I20260806 07:58:27.134025 6647 ts_tablet_manager.cc:1950] T 6047b73f65e94f58b985a178aec1e568 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.134290 6648 tablet_replica.cc:333] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.134531 6648 raft_consensus.cc:2243] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.134826 6648 pending_rounds.cc:70] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Trying to abort 1 pending ops.
I20260806 07:58:27.134940 6648 pending_rounds.cc:77] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468725617131520 op_type: NO_OP noop_request { }
I20260806 07:58:27.135090 6648 raft_consensus.cc:2889] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:27.135262 6648 raft_consensus.cc:2272] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.136376 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet b6d0ae6d095640eea8caa5853c1fbb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:27 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:27.136881 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.011s sys 0.002s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":7914,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":175,"lbm_read_time_us":759,"lbm_reads_lt_1ms":27,"lbm_write_time_us":350,"lbm_writes_lt_1ms":9,"mutex_wait_us":5023,"peak_mem_usage":32768,"reinsert_count":0,"update_count":19}
I20260806 07:58:27.137310 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 6047b73f65e94f58b985a178aec1e568 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.137809 6648 ts_tablet_manager.cc:1916] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.138840 6014 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46090:
name: "table-1"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260806 07:58:27.140372 6647 tablet_replica.cc:333] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.140585 6647 raft_consensus.cc:2243] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.140758 6647 raft_consensus.cc:2272] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.141119 6590 consensus_queue.cc:1048] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.141343 6647 ts_tablet_manager.cc:1916] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.141795 6649 ts_tablet_manager.cc:1929] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.142040 6649 log.cc:1199] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/6047b73f65e94f58b985a178aec1e568
I20260806 07:58:27.142623 6649 ts_tablet_manager.cc:1950] T 6047b73f65e94f58b985a178aec1e568 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.144688 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 6047b73f65e94f58b985a178aec1e568 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.145272 6649 tablet_replica.cc:333] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.145691 6649 raft_consensus.cc:2243] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.145939 6649 raft_consensus.cc:2272] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.146544 6649 ts_tablet_manager.cc:1916] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 07:58:27.147768 6014 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 07:58:27.150236 6648 ts_tablet_manager.cc:1929] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.150362 6648 log.cc:1199] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/1f7851b0e26d40e48bd60a7671b643b9
I20260806 07:58:27.150847 6648 ts_tablet_manager.cc:1950] T 1f7851b0e26d40e48bd60a7671b643b9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.151948 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 1f7851b0e26d40e48bd60a7671b643b9 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.154681 6648 tablet_replica.cc:333] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.156417 6647 ts_tablet_manager.cc:1929] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.156570 6647 log.cc:1199] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/6305d69bfe42420a92ca7a9fc36ec2f6
I20260806 07:58:27.156613 6623 consensus_queue.cc:1048] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.156834 6648 raft_consensus.cc:2243] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.156977 6647 ts_tablet_manager.cc:1950] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.157001 6648 raft_consensus.cc:2272] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.158082 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 6305d69bfe42420a92ca7a9fc36ec2f6 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.158290 6592 consensus_queue.cc:1048] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.159523 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.168099 6649 ts_tablet_manager.cc:1929] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.168553 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.169461 6649 log.cc:1199] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/6305d69bfe42420a92ca7a9fc36ec2f6
I20260806 07:58:27.169885 6649 ts_tablet_manager.cc:1950] T 6305d69bfe42420a92ca7a9fc36ec2f6 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.170933 6649 tablet_replica.cc:333] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.171476 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 6305d69bfe42420a92ca7a9fc36ec2f6 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.173774 6649 raft_consensus.cc:2243] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.158301 6647 tablet_replica.cc:333] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.174831 6648 ts_tablet_manager.cc:1916] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.175961 6649 raft_consensus.cc:2272] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.176842 6647 raft_consensus.cc:2243] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.177053 6647 raft_consensus.cc:2272] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.177570 6178 raft_consensus.cc:3060] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.182307 6648 ts_tablet_manager.cc:1929] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.184327 6648 log.cc:1199] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/f59dfcb1b67340e2b93fc9e8644df607
I20260806 07:58:27.184733 6648 ts_tablet_manager.cc:1950] T f59dfcb1b67340e2b93fc9e8644df607 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.186844 6647 ts_tablet_manager.cc:1916] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.187000 6568 consensus_queue.cc:1048] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.185861 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet f59dfcb1b67340e2b93fc9e8644df607 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.189145 6648 tablet_replica.cc:333] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.189411 6648 raft_consensus.cc:2243] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.190100 6648 raft_consensus.cc:2272] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.191104 6648 ts_tablet_manager.cc:1916] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.194823 6649 ts_tablet_manager.cc:1916] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.201673 6649 ts_tablet_manager.cc:1929] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.201881 6649 log.cc:1199] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/1f7851b0e26d40e48bd60a7671b643b9
I20260806 07:58:27.202291 6649 ts_tablet_manager.cc:1950] T 1f7851b0e26d40e48bd60a7671b643b9 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.203401 6649 tablet_replica.cc:333] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.203631 6649 raft_consensus.cc:2243] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.203917 6649 pending_rounds.cc:70] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Trying to abort 1 pending ops.
I20260806 07:58:27.204012 6649 pending_rounds.cc:77] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468725532762112 op_type: NO_OP noop_request { }
I20260806 07:58:27.204167 6649 raft_consensus.cc:2889] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:27.204312 6649 raft_consensus.cc:2272] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.205271 6649 ts_tablet_manager.cc:1916] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.205524 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:27.205928 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:27.206226 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.206523 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 1f7851b0e26d40e48bd60a7671b643b9 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.206795 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20260806 07:58:27.207216 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.209724 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.210214 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.210830 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.211185 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:27.211949 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260806 07:58:27.212335 6649 ts_tablet_manager.cc:1929] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.212495 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.212567 6649 log.cc:1199] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/f59dfcb1b67340e2b93fc9e8644df607
I20260806 07:58:27.213179 6649 ts_tablet_manager.cc:1950] T f59dfcb1b67340e2b93fc9e8644df607 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.213161 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.214735 6647 ts_tablet_manager.cc:1929] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.214811 6146 tablet_service.cc:1511] Processing CreateTablet for tablet 4e093658a8714105a67c0165bc2978b1 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.214735 6155 tablet_service.cc:1511] Processing CreateTablet for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.209532 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.010s sys 0.000s Metrics: {"bytes_written":6937,"cfile_init":1,"dirs.queue_time_us":101,"dirs.run_cpu_time_us":109,"dirs.run_wall_time_us":547,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1582,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260806 07:58:27.215274 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.205470 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.011s sys 0.000s Metrics: {"bytes_written":6937,"cfile_init":1,"dirs.queue_time_us":812,"dirs.run_cpu_time_us":150,"dirs.run_wall_time_us":1388,"drs_written":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3342,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":17}
I20260806 07:58:27.215478 6160 tablet_service.cc:1511] Processing CreateTablet for tablet 0457d931c271465e8d3c17461af639a5 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.215946 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0457d931c271465e8d3c17461af639a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.216095 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.216437 6647 log.cc:1199] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/1f7851b0e26d40e48bd60a7671b643b9
I20260806 07:58:27.216852 6647 ts_tablet_manager.cc:1950] T 1f7851b0e26d40e48bd60a7671b643b9 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.217907 6647 tablet_replica.cc:333] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.218163 6647 raft_consensus.cc:2243] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.218376 6647 raft_consensus.cc:2272] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.218784 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet f59dfcb1b67340e2b93fc9e8644df607 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.219174 6647 ts_tablet_manager.cc:1916] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.219853 6433 tablet_service.cc:1511] Processing CreateTablet for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.220346 6428 tablet_service.cc:1511] Processing CreateTablet for tablet 4e093658a8714105a67c0165bc2978b1 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.220727 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e093658a8714105a67c0165bc2978b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.222302 6537 tablet_bootstrap.cc:492] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.224198 6537 tablet_bootstrap.cc:654] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.224735 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 1f7851b0e26d40e48bd60a7671b643b9 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.225729 6647 ts_tablet_manager.cc:1929] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:27.225930 6647 log.cc:1199] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/f59dfcb1b67340e2b93fc9e8644df607
I20260806 07:58:27.226389 6647 ts_tablet_manager.cc:1950] T f59dfcb1b67340e2b93fc9e8644df607 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.226619 6293 tablet_service.cc:1511] Processing CreateTablet for tablet 0457d931c271465e8d3c17461af639a5 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.226912 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.583905
I20260806 07:58:27.227020 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0457d931c271465e8d3c17461af639a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.227587 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.583905
I20260806 07:58:27.227643 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet f59dfcb1b67340e2b93fc9e8644df607 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.228152 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.583905
I20260806 07:58:27.228175 6647 tablet_replica.cc:333] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.228431 6647 raft_consensus.cc:2243] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.228696 6647 raft_consensus.cc:2272] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.229825 6294 tablet_service.cc:1511] Processing CreateTablet for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.230232 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.230240 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.005s sys 0.000s Metrics: {"bytes_written":702,"delete_count":0,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260806 07:58:27.231671 6572 consensus_queue.cc:1048] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.234251 6532 tablet_bootstrap.cc:492] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.234186 6296 tablet_service.cc:1511] Processing CreateTablet for tablet 4e093658a8714105a67c0165bc2978b1 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.234560 6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e093658a8714105a67c0165bc2978b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.236334 6532 tablet_bootstrap.cc:654] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.237314 6649 tablet_replica.cc:333] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.237726 6649 raft_consensus.cc:2243] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.238360 6649 raft_consensus.cc:2272] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.239607 6649 ts_tablet_manager.cc:1916] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.241568 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e093658a8714105a67c0165bc2978b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.242445 6581 consensus_queue.cc:1048] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.242599 6433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.242998 6537 tablet_bootstrap.cc:492] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.243232 6537 ts_tablet_manager.cc:1403] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.004s
I20260806 07:58:27.246383 6568 consensus_queue.cc:1048] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.246678 6537 raft_consensus.cc:359] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.247031 6537 raft_consensus.cc:385] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.247165 6537 raft_consensus.cc:740] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.247556 6537 consensus_queue.cc:260] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.248373 6537 ts_tablet_manager.cc:1434] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:27.251847 6649 ts_tablet_manager.cc:1929] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.252034 6649 log.cc:1199] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/4d7d632b15694509ac4cce794498fbb3
I20260806 07:58:27.252545 6649 ts_tablet_manager.cc:1950] T 4d7d632b15694509ac4cce794498fbb3 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.253755 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 4d7d632b15694509ac4cce794498fbb3 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.254168 6649 tablet_replica.cc:333] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.254230 6002 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:27.254422 6649 raft_consensus.cc:2243] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.254681 6649 raft_consensus.cc:2272] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.256290 6424 tablet_service.cc:1511] Processing CreateTablet for tablet 0457d931c271465e8d3c17461af639a5 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.256723 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0457d931c271465e8d3c17461af639a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.256976 6155 tablet_service.cc:1511] Processing CreateTablet for tablet 46f5138fc13a493bb7217f123071c20a (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.257503 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46f5138fc13a493bb7217f123071c20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.258368 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 46f5138fc13a493bb7217f123071c20a (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.258749 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46f5138fc13a493bb7217f123071c20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.262326 6432 tablet_service.cc:1511] Processing CreateTablet for tablet 88ff8d074e9e4a7d9f788fb501528bae (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.262694 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ff8d074e9e4a7d9f788fb501528bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.264017 6149 tablet_service.cc:1511] Processing CreateTablet for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.264408 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.264768 6538 consensus_queue.cc:1048] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260806 07:58:27.268486 6429 tablet_service.cc:1511] Processing CreateTablet for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.268890 6429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.271273 6421 tablet_service.cc:1511] Processing CreateTablet for tablet c0e540b33899435da8546e5080bfa40b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.271667 6421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e540b33899435da8546e5080bfa40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.271766 6286 tablet_service.cc:1511] Processing CreateTablet for tablet 88ff8d074e9e4a7d9f788fb501528bae (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.272223 6286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ff8d074e9e4a7d9f788fb501528bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.272295 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.003s sys 0.000s Metrics: {"bytes_written":601,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":10}
I20260806 07:58:27.272910 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.278419 6531 tablet_bootstrap.cc:492] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.280737 6531 tablet_bootstrap.cc:654] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.280804 6572 consensus_queue.cc:1048] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260806 07:58:27.280987 6537 tablet_bootstrap.cc:492] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.281247 6647 ts_tablet_manager.cc:1916] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.281505 6295 tablet_service.cc:1511] Processing CreateTablet for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.283004 6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.281682 6538 consensus_queue.cc:1048] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260806 07:58:27.282510 6283 tablet_service.cc:1511] Processing CreateTablet for tablet c0e540b33899435da8546e5080bfa40b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.289172 6423 tablet_service.cc:1511] Processing CreateTablet for tablet f89b495f1733475895ce958ae57861e1 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.290249 6649 ts_tablet_manager.cc:1916] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.290947 6283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e540b33899435da8546e5080bfa40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.292227 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.082s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":81633}
I20260806 07:58:27.292479 6537 tablet_bootstrap.cc:654] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.292582 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.076s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":76201}
I20260806 07:58:27.292822 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.083s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":82726}
I20260806 07:58:27.293504 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.561465
I20260806 07:58:27.293888 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.561465
I20260806 07:58:27.294209 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.561465
I20260806 07:58:27.296022 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.296142 6284 tablet_service.cc:1511] Processing CreateTablet for tablet f89b495f1733475895ce958ae57861e1 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.296371 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.296518 6284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89b495f1733475895ce958ae57861e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.298396 6146 tablet_service.cc:1511] Processing CreateTablet for tablet 88ff8d074e9e4a7d9f788fb501528bae (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.298590 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.476255
I20260806 07:58:27.298921 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ff8d074e9e4a7d9f788fb501528bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.299080 6288 tablet_service.cc:1511] Processing CreateTablet for tablet 59cd956622e64e7fbfd27f370d63605b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.299198 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.299469 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cd956622e64e7fbfd27f370d63605b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.300185 6149 tablet_service.cc:1511] Processing CreateTablet for tablet aacb42bf87304fbe8aa1edf4cc6a9e58 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.300552 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacb42bf87304fbe8aa1edf4cc6a9e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.301841 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.302271 6581 consensus_queue.cc:1048] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.303133 6648 ts_tablet_manager.cc:1929] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.303254 6648 log.cc:1199] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/4d7d632b15694509ac4cce794498fbb3
I20260806 07:58:27.290369 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89b495f1733475895ce958ae57861e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.304081 6424 tablet_service.cc:1511] Processing CreateTablet for tablet 59cd956622e64e7fbfd27f370d63605b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.310019 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cd956622e64e7fbfd27f370d63605b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.313437 6433 tablet_service.cc:1511] Processing CreateTablet for tablet 8961173cf8d24b8eb022e994f90970da (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.313861 6433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8961173cf8d24b8eb022e994f90970da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.314880 6432 tablet_service.cc:1511] Processing CreateTablet for tablet aacb42bf87304fbe8aa1edf4cc6a9e58 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.315402 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacb42bf87304fbe8aa1edf4cc6a9e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.318485 6282 tablet_service.cc:1511] Processing CreateTablet for tablet 8961173cf8d24b8eb022e994f90970da (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.319011 6282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8961173cf8d24b8eb022e994f90970da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.321161 6285 tablet_service.cc:1511] Processing CreateTablet for tablet aacb42bf87304fbe8aa1edf4cc6a9e58 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.321645 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacb42bf87304fbe8aa1edf4cc6a9e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.323007 6568 consensus_queue.cc:1048] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.325111 6590 consensus_queue.cc:1048] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 07:58:27.325891 6293 tablet_service.cc:1511] Processing CreateTablet for tablet 46f5138fc13a493bb7217f123071c20a (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.326725 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46f5138fc13a493bb7217f123071c20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.326947 6148 tablet_service.cc:1511] Processing CreateTablet for tablet 59cd956622e64e7fbfd27f370d63605b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.327416 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cd956622e64e7fbfd27f370d63605b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.327867 6287 tablet_service.cc:1511] Processing CreateTablet for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.328199 6152 tablet_service.cc:1511] Processing CreateTablet for tablet f89b495f1733475895ce958ae57861e1 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.328336 6156 tablet_service.cc:1511] Processing CreateTablet for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.328718 6152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89b495f1733475895ce958ae57861e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.329108 6156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.329530 6155 tablet_service.cc:1511] Processing CreateTablet for tablet 8961173cf8d24b8eb022e994f90970da (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.329911 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8961173cf8d24b8eb022e994f90970da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.330633 6648 ts_tablet_manager.cc:1950] T 4d7d632b15694509ac4cce794498fbb3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.332170 6648 tablet_replica.cc:333] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.332186 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 4d7d632b15694509ac4cce794498fbb3 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.332403 6648 raft_consensus.cc:2243] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.332597 6648 raft_consensus.cc:2272] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.333425 6648 ts_tablet_manager.cc:1916] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.338706 6537 tablet_bootstrap.cc:492] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.338588 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.006s sys 0.000s Metrics: {"bytes_written":518,"delete_count":0,"lbm_write_time_us":175,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":6}
I20260806 07:58:27.338888 6537 ts_tablet_manager.cc:1403] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.002s
I20260806 07:58:27.339939 6537 raft_consensus.cc:359] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.340371 6537 raft_consensus.cc:385] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.340505 6537 raft_consensus.cc:740] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.340847 6537 consensus_queue.cc:260] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.341509 6537 ts_tablet_manager.cc:1434] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:27.341902 6537 tablet_bootstrap.cc:492] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.347167 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.110s user 0.011s sys 0.000s Metrics: {"dirs.queue_time_us":4306,"dirs.run_cpu_time_us":904,"dirs.run_wall_time_us":4754,"lbm_write_time_us":301,"lbm_writes_lt_1ms":7,"mutex_wait_us":814,"num_input_rowsets":0}
I20260806 07:58:27.352943 6649 ts_tablet_manager.cc:1929] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.353111 6649 log.cc:1199] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/1d800927238145aea3e3c82d1bf158e1
I20260806 07:58:27.353456 6531 tablet_bootstrap.cc:492] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.353614 6649 ts_tablet_manager.cc:1950] T 1d800927238145aea3e3c82d1bf158e1 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.353763 6531 ts_tablet_manager.cc:1403] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.009s
I20260806 07:58:27.354715 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 1d800927238145aea3e3c82d1bf158e1 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.355134 6649 tablet_replica.cc:333] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.355335 6649 raft_consensus.cc:2243] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.355551 6649 raft_consensus.cc:2272] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.355983 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.012s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":1998,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":20567,"dirs.run_cpu_time_us":127,"dirs.run_wall_time_us":579,"lbm_read_time_us":838,"lbm_reads_lt_1ms":27,"lbm_write_time_us":398,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":16}
I20260806 07:58:27.357689 6649 ts_tablet_manager.cc:1916] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.358963 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.359378 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.354736 6531 raft_consensus.cc:359] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.359530 6148 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.359934 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee24a9ece7f461a96e7f386b89c6208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.358623 5930 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:27.363848 6531 raft_consensus.cc:385] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.364034 6531 raft_consensus.cc:740] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.364405 6531 consensus_queue.cc:260] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.365159 6531 ts_tablet_manager.cc:1434] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260806 07:58:27.366676 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.067s user 0.014s sys 0.000s Metrics: {"dirs.queue_time_us":1951,"dirs.run_cpu_time_us":860,"dirs.run_wall_time_us":6420,"lbm_write_time_us":322,"lbm_writes_lt_1ms":7,"num_input_rowsets":0,"spinlock_wait_cycles":316288,"thread_start_us":46682,"threads_started":1}
I20260806 07:58:27.367254 6531 tablet_bootstrap.cc:492] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.369267 6531 tablet_bootstrap.cc:654] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.370061 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.491919
I20260806 07:58:27.370657 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.491919
I20260806 07:58:27.371176 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.491919
I20260806 07:58:27.371722 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.491919
I20260806 07:58:27.384799 6003 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:27.386164 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1922 bytes on disk
I20260806 07:58:27.386488 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1922 bytes on disk
I20260806 07:58:27.386927 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4}
I20260806 07:58:27.387660 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.393023 6149 tablet_service.cc:1511] Processing CreateTablet for tablet 06cc4788571140b2ab78811966d2e06d (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.379806 6649 ts_tablet_manager.cc:1929] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.383628 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.393453 6649 log.cc:1199] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/727a2816995c4456b2210e7ad8913e3c
I20260806 07:58:27.393684 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee24a9ece7f461a96e7f386b89c6208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.394109 6649 ts_tablet_manager.cc:1950] T 727a2816995c4456b2210e7ad8913e3c P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.395239 6649 tablet_replica.cc:333] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.395550 6649 raft_consensus.cc:2243] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.395830 6649 raft_consensus.cc:2272] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.397827 5928 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:27.390450 6285 tablet_service.cc:1511] Processing CreateTablet for tablet 462655eb372b4284a2b88378bbb0f686 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.398237 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 727a2816995c4456b2210e7ad8913e3c (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.398307 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 462655eb372b4284a2b88378bbb0f686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.402093 6284 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.329779 6160 tablet_service.cc:1511] Processing CreateTablet for tablet c0e540b33899435da8546e5080bfa40b (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.403191 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e540b33899435da8546e5080bfa40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.404424 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.176s user 0.029s sys 0.001s Metrics: {"bytes_written":18784,"cfile_cache_hit":16,"cfile_cache_hit_bytes":9380,"cfile_cache_miss":42,"cfile_cache_miss_bytes":15285,"cfile_init":12,"delta_iterators_relevant":13,"dirs.queue_time_us":33336,"dirs.run_cpu_time_us":1481,"dirs.run_wall_time_us":8099,"drs_written":1,"lbm_read_time_us":3073,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":89,"lbm_write_time_us":961,"lbm_writes_lt_1ms":27,"mutex_wait_us":5079,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":37,"thread_start_us":14917,"threads_started":1}
I20260806 07:58:27.406020 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.000s sys 0.011s Metrics: {"dirs.queue_time_us":2074,"dirs.run_cpu_time_us":764,"dirs.run_wall_time_us":4887,"lbm_write_time_us":329,"lbm_writes_lt_1ms":7,"mutex_wait_us":233,"num_input_rowsets":0}
I20260806 07:58:27.411128 6537 tablet_bootstrap.cc:654] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.424274 6648 ts_tablet_manager.cc:1929] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.424458 6648 log.cc:1199] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/1d800927238145aea3e3c82d1bf158e1
I20260806 07:58:27.425429 6649 ts_tablet_manager.cc:1916] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.428084 6592 consensus_queue.cc:1048] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.433255 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1922 bytes on disk
I20260806 07:58:27.434149 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":162,"lbm_reads_lt_1ms":4}
I20260806 07:58:27.436293 6531 tablet_bootstrap.cc:492] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.436483 6531 ts_tablet_manager.cc:1403] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.069s user 0.014s sys 0.005s
I20260806 07:58:27.437332 6531 raft_consensus.cc:359] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.437528 6531 raft_consensus.cc:385] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.437666 6531 raft_consensus.cc:740] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.437960 6531 consensus_queue.cc:260] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.438163 6648 ts_tablet_manager.cc:1950] T 1d800927238145aea3e3c82d1bf158e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.438647 6531 ts_tablet_manager.cc:1434] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 07:58:27.438925 6531 tablet_bootstrap.cc:492] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.439601 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06cc4788571140b2ab78811966d2e06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.440164 6623 raft_consensus.cc:493] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.440356 6623 raft_consensus.cc:515] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.441006 6623 leader_election.cc:290] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.442306 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e093658a8714105a67c0165bc2978b1" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:27.442855 6384 leader_election.cc:343] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.443254 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e093658a8714105a67c0165bc2978b1" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.443468 6177 raft_consensus.cc:2468] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:27.390443 6432 tablet_service.cc:1511] Processing CreateTablet for tablet 462655eb372b4284a2b88378bbb0f686 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.443877 6387 leader_election.cc:304] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:27.444244 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 462655eb372b4284a2b88378bbb0f686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.444337 6623 raft_consensus.cc:2804] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.444444 6623 raft_consensus.cc:493] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.444604 6623 raft_consensus.cc:3060] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.450454 6288 tablet_service.cc:1511] Processing CreateTablet for tablet 06cc4788571140b2ab78811966d2e06d (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.450618 6623 raft_consensus.cc:515] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.451670 6623 leader_election.cc:290] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.451979 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 06cc4788571140b2ab78811966d2e06d (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.452404 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06cc4788571140b2ab78811966d2e06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.452406 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e093658a8714105a67c0165bc2978b1" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:27.452942 6384 leader_election.cc:343] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.453898 6532 tablet_bootstrap.cc:492] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.454095 6532 ts_tablet_manager.cc:1403] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.220s user 0.002s sys 0.006s
I20260806 07:58:27.454941 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e093658a8714105a67c0165bc2978b1" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:27.455013 6532 raft_consensus.cc:359] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.455220 6532 raft_consensus.cc:385] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.455310 6532 raft_consensus.cc:740] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.455276 6180 raft_consensus.cc:3060] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.458534 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06cc4788571140b2ab78811966d2e06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.459012 6287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.459038 5931 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:27.461771 6284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee24a9ece7f461a96e7f386b89c6208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.463140 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.079s user 0.008s sys 0.000s Metrics: {"dirs.queue_time_us":4313,"dirs.run_cpu_time_us":662,"dirs.run_wall_time_us":4841,"lbm_write_time_us":303,"lbm_writes_lt_1ms":7,"mutex_wait_us":3265,"num_input_rowsets":0}
I20260806 07:58:27.469789 6532 consensus_queue.cc:260] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.470443 6532 ts_tablet_manager.cc:1434] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.016s user 0.000s sys 0.002s
I20260806 07:58:27.470830 6532 tablet_bootstrap.cc:492] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.471567 6148 tablet_service.cc:1511] Processing CreateTablet for tablet 81dd62212ec041f19f9760efb21ded44 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.471980 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81dd62212ec041f19f9760efb21ded44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.472697 6532 tablet_bootstrap.cc:654] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.475013 6531 tablet_bootstrap.cc:654] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.481503 6647 ts_tablet_manager.cc:1929] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.483167 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 1d800927238145aea3e3c82d1bf158e1 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.483639 6648 tablet_replica.cc:333] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.483843 6648 raft_consensus.cc:2243] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.487444 6648 raft_consensus.cc:2272] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.488240 6648 ts_tablet_manager.cc:1916] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.489076 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.117s user 0.025s sys 0.011s Metrics: {"bytes_written":18784,"cfile_cache_hit":17,"cfile_cache_hit_bytes":13208,"cfile_cache_miss":43,"cfile_cache_miss_bytes":19722,"cfile_init":12,"delta_iterators_relevant":14,"dirs.queue_time_us":13067,"dirs.run_cpu_time_us":1037,"dirs.run_wall_time_us":6102,"drs_written":1,"lbm_read_time_us":2233,"lbm_reads_lt_1ms":91,"lbm_write_time_us":927,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":37,"thread_start_us":50394,"threads_started":2}
I20260806 07:58:27.493568 6424 tablet_service.cc:1511] Processing CreateTablet for tablet 81dd62212ec041f19f9760efb21ded44 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.494050 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81dd62212ec041f19f9760efb21ded44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.492098 6293 tablet_service.cc:1511] Processing CreateTablet for tablet 81dd62212ec041f19f9760efb21ded44 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.496677 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81dd62212ec041f19f9760efb21ded44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.500622 6647 log.cc:1199] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/4d7d632b15694509ac4cce794498fbb3
I20260806 07:58:27.501240 6647 ts_tablet_manager.cc:1950] T 4d7d632b15694509ac4cce794498fbb3 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.510172 6649 ts_tablet_manager.cc:1929] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.510339 6649 log.cc:1199] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/8d6639b2db4f4cb99f33465ace8f0294
I20260806 07:58:27.510834 6649 ts_tablet_manager.cc:1950] T 8d6639b2db4f4cb99f33465ace8f0294 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.511412 5930 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:27.511993 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 8d6639b2db4f4cb99f33465ace8f0294 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.515784 6146 tablet_service.cc:1511] Processing CreateTablet for tablet 462655eb372b4284a2b88378bbb0f686 (DEFAULT_TABLE table=table-1 [id=d056973627284d368d205bef9293abd6]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:27.516177 6180 raft_consensus.cc:2468] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:27.516321 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 462655eb372b4284a2b88378bbb0f686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:27.516754 6387 leader_election.cc:304] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:27.517277 6623 raft_consensus.cc:2804] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.517498 6623 raft_consensus.cc:697] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:27.517843 6623 consensus_queue.cc:237] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.522709 6537 tablet_bootstrap.cc:492] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.522956 6537 ts_tablet_manager.cc:1403] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.181s user 0.006s sys 0.006s
I20260806 07:58:27.523869 6537 raft_consensus.cc:359] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.524147 6537 raft_consensus.cc:385] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.524271 6537 raft_consensus.cc:740] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.524623 6537 consensus_queue.cc:260] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.526788 6647 tablet_replica.cc:333] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.527065 6647 raft_consensus.cc:2243] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.527468 6537 ts_tablet_manager.cc:1434] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260806 07:58:27.527693 6647 raft_consensus.cc:2272] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.527938 6537 tablet_bootstrap.cc:492] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.529812 6537 tablet_bootstrap.cc:654] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.530776 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 4d7d632b15694509ac4cce794498fbb3 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.533957 6647 ts_tablet_manager.cc:1916] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.538982 6649 tablet_replica.cc:333] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.539292 6649 raft_consensus.cc:2243] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.539739 6649 raft_consensus.cc:2272] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.540120 6531 tablet_bootstrap.cc:492] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.540365 6531 ts_tablet_manager.cc:1403] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.101s user 0.019s sys 0.012s
I20260806 07:58:27.541152 6649 ts_tablet_manager.cc:1916] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.541687 6531 raft_consensus.cc:359] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.542255 6531 raft_consensus.cc:385] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.542312 6647 ts_tablet_manager.cc:1929] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.542347 6531 raft_consensus.cc:740] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.542474 6647 log.cc:1199] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/1d800927238145aea3e3c82d1bf158e1
I20260806 07:58:27.538677 6648 ts_tablet_manager.cc:1929] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.542699 6648 log.cc:1199] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/727a2816995c4456b2210e7ad8913e3c
I20260806 07:58:27.542630 6531 consensus_queue.cc:260] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.542910 6647 ts_tablet_manager.cc:1950] T 1d800927238145aea3e3c82d1bf158e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.543221 6531 ts_tablet_manager.cc:1434] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:27.543577 6531 tablet_bootstrap.cc:492] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.543871 6648 ts_tablet_manager.cc:1950] T 727a2816995c4456b2210e7ad8913e3c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.544091 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 1d800927238145aea3e3c82d1bf158e1 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.545444 6531 tablet_bootstrap.cc:654] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.545471 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 727a2816995c4456b2210e7ad8913e3c (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.547477 6532 tablet_bootstrap.cc:492] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.547636 6532 ts_tablet_manager.cc:1403] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.077s user 0.000s sys 0.030s
I20260806 07:58:27.547796 6648 tablet_replica.cc:333] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.548054 6648 raft_consensus.cc:2243] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.548290 6648 raft_consensus.cc:2272] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.548462 6532 raft_consensus.cc:359] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.548662 6532 raft_consensus.cc:385] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.548799 6532 raft_consensus.cc:740] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.549134 6532 consensus_queue.cc:260] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.549538 6647 tablet_replica.cc:333] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.549698 6648 ts_tablet_manager.cc:1916] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.549808 6647 raft_consensus.cc:2243] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.549736 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.316s user 0.009s sys 0.011s Metrics: {"dirs.queue_time_us":29673,"dirs.run_cpu_time_us":921,"dirs.run_wall_time_us":12675,"lbm_write_time_us":358,"lbm_writes_lt_1ms":7,"mutex_wait_us":2165,"num_input_rowsets":0,"spinlock_wait_cycles":14208,"thread_start_us":118267,"threads_started":1}
I20260806 07:58:27.550009 6647 raft_consensus.cc:2272] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.553949 6647 ts_tablet_manager.cc:1916] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.554865 6016 catalog_manager.cc:5697] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:27.556625 6537 tablet_bootstrap.cc:492] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.556841 6537 ts_tablet_manager.cc:1403] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20260806 07:58:27.557245 6532 ts_tablet_manager.cc:1434] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260806 07:58:27.557842 6537 raft_consensus.cc:359] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.558105 6537 raft_consensus.cc:385] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.558176 6532 tablet_bootstrap.cc:492] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.558302 6537 raft_consensus.cc:740] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.559192 6537 consensus_queue.cc:260] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.559954 6532 tablet_bootstrap.cc:654] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.561700 6537 ts_tablet_manager.cc:1434] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 07:58:27.561941 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.562063 6537 tablet_bootstrap.cc:492] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.562279 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.562587 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.562856 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.564284 6531 tablet_bootstrap.cc:492] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.564433 6531 ts_tablet_manager.cc:1403] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260806 07:58:27.564905 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.565199 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.565431 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.565371 6531 raft_consensus.cc:359] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.565630 6531 raft_consensus.cc:385] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.565742 6531 raft_consensus.cc:740] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.566062 6531 consensus_queue.cc:260] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.566766 6531 ts_tablet_manager.cc:1434] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:27.567035 6531 tablet_bootstrap.cc:492] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.568068 6649 ts_tablet_manager.cc:1929] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.568214 6649 log.cc:1199] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/ff476732379d406e8093e43da857d681
I20260806 07:58:27.568760 6649 ts_tablet_manager.cc:1950] T ff476732379d406e8093e43da857d681 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.568964 6531 tablet_bootstrap.cc:654] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.569972 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet ff476732379d406e8093e43da857d681 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.571955 6532 tablet_bootstrap.cc:492] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.572144 6532 ts_tablet_manager.cc:1403] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
I20260806 07:58:27.572393 6647 ts_tablet_manager.cc:1929] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.572540 6647 log.cc:1199] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/727a2816995c4456b2210e7ad8913e3c
I20260806 07:58:27.573115 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.573143 6532 raft_consensus.cc:359] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.573354 6532 raft_consensus.cc:385] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.573442 6532 raft_consensus.cc:740] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.573463 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.573776 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.573736 6532 consensus_queue.cc:260] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.574018 6537 tablet_bootstrap.cc:654] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.574265 6532 ts_tablet_manager.cc:1434] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:27.574601 6532 tablet_bootstrap.cc:492] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.575449 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.575791 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.576124 6648 ts_tablet_manager.cc:1929] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.576277 6648 log.cc:1199] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/8d6639b2db4f4cb99f33465ace8f0294
I20260806 07:58:27.576459 6532 tablet_bootstrap.cc:654] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.576638 6647 ts_tablet_manager.cc:1950] T 727a2816995c4456b2210e7ad8913e3c P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.576741 6648 ts_tablet_manager.cc:1950] T 8d6639b2db4f4cb99f33465ace8f0294 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.577836 6649 tablet_replica.cc:333] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.577797 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 727a2816995c4456b2210e7ad8913e3c (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.578083 6649 raft_consensus.cc:2243] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.578287 6649 raft_consensus.cc:2272] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.579111 6647 tablet_replica.cc:333] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.579346 6647 raft_consensus.cc:2243] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.579699 6647 raft_consensus.cc:2272] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.580922 6647 ts_tablet_manager.cc:1916] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.583261 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.191s user 0.009s sys 0.000s Metrics: {"dirs.queue_time_us":477,"dirs.run_cpu_time_us":918,"dirs.run_wall_time_us":5697,"lbm_write_time_us":321,"lbm_writes_lt_1ms":7,"num_input_rowsets":0,"spinlock_wait_cycles":3968}
I20260806 07:58:27.581046 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.018s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.586215 6649 ts_tablet_manager.cc:1916] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.586514 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"mutex_wait_us":7529,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.587819 6647 ts_tablet_manager.cc:1929] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.587982 6647 log.cc:1199] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/8d6639b2db4f4cb99f33465ace8f0294
I20260806 07:58:27.588426 6647 ts_tablet_manager.cc:1950] T 8d6639b2db4f4cb99f33465ace8f0294 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.589195 6532 tablet_bootstrap.cc:492] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.589455 6647 tablet_replica.cc:333] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.589689 6647 raft_consensus.cc:2243] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.589669 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 8d6639b2db4f4cb99f33465ace8f0294 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.589887 6647 raft_consensus.cc:2272] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.589948 6532 ts_tablet_manager.cc:1403] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.002s
I20260806 07:58:27.590811 6647 ts_tablet_manager.cc:1916] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.591621 6532 raft_consensus.cc:359] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.591836 6532 raft_consensus.cc:385] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.591948 6532 raft_consensus.cc:740] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.592415 6532 consensus_queue.cc:260] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.594592 6649 ts_tablet_manager.cc:1929] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.594751 6649 log.cc:1199] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/0d04186bb02946928cc22152266d8a3f
I20260806 07:58:27.595217 6649 ts_tablet_manager.cc:1950] T 0d04186bb02946928cc22152266d8a3f P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.596009 6532 ts_tablet_manager.cc:1434] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 07:58:27.596279 6649 tablet_replica.cc:333] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.596333 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 0d04186bb02946928cc22152266d8a3f (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.596508 6649 raft_consensus.cc:2243] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.596877 6649 raft_consensus.cc:2272] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.597093 6531 tablet_bootstrap.cc:492] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.597303 6531 ts_tablet_manager.cc:1403] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.005s
I20260806 07:58:27.598203 6531 raft_consensus.cc:359] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.598295 6647 ts_tablet_manager.cc:1929] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.598413 6647 log.cc:1199] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/ff476732379d406e8093e43da857d681
I20260806 07:58:27.598723 6531 raft_consensus.cc:385] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.598822 6531 raft_consensus.cc:740] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.598866 6648 tablet_replica.cc:333] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.599009 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 8d6639b2db4f4cb99f33465ace8f0294 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.599076 6648 raft_consensus.cc:2243] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.599141 6531 consensus_queue.cc:260] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.599320 6648 raft_consensus.cc:2272] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.599633 6647 ts_tablet_manager.cc:1950] T ff476732379d406e8093e43da857d681 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.600340 6537 tablet_bootstrap.cc:492] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.600525 6537 ts_tablet_manager.cc:1403] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.000s
I20260806 07:58:27.600659 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet ff476732379d406e8093e43da857d681 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.601352 6537 raft_consensus.cc:359] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.601565 6537 raft_consensus.cc:385] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.601678 6537 raft_consensus.cc:740] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.601966 6537 consensus_queue.cc:260] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.602386 6531 ts_tablet_manager.cc:1434] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 07:58:27.602488 6537 ts_tablet_manager.cc:1434] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:27.602813 6537 tablet_bootstrap.cc:492] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.603431 6531 tablet_bootstrap.cc:492] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.604673 6537 tablet_bootstrap.cc:654] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.605202 6647 tablet_replica.cc:333] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.605479 6647 raft_consensus.cc:2243] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.605754 6647 raft_consensus.cc:2272] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.606452 6648 ts_tablet_manager.cc:1916] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.607656 6532 tablet_bootstrap.cc:492] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.609865 6647 ts_tablet_manager.cc:1916] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.610522 6532 tablet_bootstrap.cc:654] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.612185 6531 tablet_bootstrap.cc:654] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.612679 6537 tablet_bootstrap.cc:492] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.612890 6537 ts_tablet_manager.cc:1403] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260806 07:58:27.613376 6648 ts_tablet_manager.cc:1929] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.613533 6648 log.cc:1199] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/ff476732379d406e8093e43da857d681
I20260806 07:58:27.614138 6648 ts_tablet_manager.cc:1950] T ff476732379d406e8093e43da857d681 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.614044 6537 raft_consensus.cc:359] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.614391 6537 raft_consensus.cc:385] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.614543 6537 raft_consensus.cc:740] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.614928 6537 consensus_queue.cc:260] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.615483 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet ff476732379d406e8093e43da857d681 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.616040 6648 tablet_replica.cc:333] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.616380 6648 raft_consensus.cc:2243] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.616811 6648 raft_consensus.cc:2272] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.617933 6648 ts_tablet_manager.cc:1916] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.618885 6537 ts_tablet_manager.cc:1434] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260806 07:58:27.619616 6649 ts_tablet_manager.cc:1916] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.620436 6647 ts_tablet_manager.cc:1929] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.620590 6647 log.cc:1199] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/0d04186bb02946928cc22152266d8a3f
I20260806 07:58:27.621068 6647 ts_tablet_manager.cc:1950] T 0d04186bb02946928cc22152266d8a3f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.622244 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 0d04186bb02946928cc22152266d8a3f (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.622676 6647 tablet_replica.cc:333] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.622891 6647 raft_consensus.cc:2243] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.623051 6532 tablet_bootstrap.cc:492] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.623118 6647 raft_consensus.cc:2272] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.623240 6532 ts_tablet_manager.cc:1403] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.003s
I20260806 07:58:27.624080 6532 raft_consensus.cc:359] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.624321 6532 raft_consensus.cc:385] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.624411 6532 raft_consensus.cc:740] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.624519 6537 tablet_bootstrap.cc:492] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.624178 6647 ts_tablet_manager.cc:1916] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.626279 6648 ts_tablet_manager.cc:1929] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.626495 6648 log.cc:1199] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/0d04186bb02946928cc22152266d8a3f
I20260806 07:58:27.626495 6572 raft_consensus.cc:493] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.626660 6572 raft_consensus.cc:515] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.627103 6648 ts_tablet_manager.cc:1950] T 0d04186bb02946928cc22152266d8a3f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.627389 6572 leader_election.cc:290] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.627818 6537 tablet_bootstrap.cc:654] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.628158 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e540b33899435da8546e5080bfa40b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.628376 6453 raft_consensus.cc:2468] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:27.628984 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e540b33899435da8546e5080bfa40b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.629935 6250 leader_election.cc:304] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
W20260806 07:58:27.630337 6252 leader_election.cc:343] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.630735 6572 raft_consensus.cc:2804] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.630826 6572 raft_consensus.cc:493] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.630920 6572 raft_consensus.cc:3060] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.634274 6572 raft_consensus.cc:515] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.634672 6647 ts_tablet_manager.cc:1929] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.634657 6649 ts_tablet_manager.cc:1929] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.634800 6647 log.cc:1199] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/8d71ed2aa2bf43ffbc98dda21fc73eac
I20260806 07:58:27.634832 6649 log.cc:1199] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/8d71ed2aa2bf43ffbc98dda21fc73eac
I20260806 07:58:27.635274 6647 ts_tablet_manager.cc:1950] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.635430 6649 ts_tablet_manager.cc:1950] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.635860 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e540b33899435da8546e5080bfa40b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:27.636322 6647 tablet_replica.cc:333] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.636518 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 8d71ed2aa2bf43ffbc98dda21fc73eac (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.636550 6647 raft_consensus.cc:2243] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
W20260806 07:58:27.636567 6252 leader_election.cc:343] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.636690 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 8d71ed2aa2bf43ffbc98dda21fc73eac (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.636916 6647 raft_consensus.cc:2272] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.637058 6649 tablet_replica.cc:333] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.637368 6649 raft_consensus.cc:2243] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.637337 6532 consensus_queue.cc:260] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.637573 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e540b33899435da8546e5080bfa40b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:27.637770 6453 raft_consensus.cc:3060] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.637933 6532 ts_tablet_manager.cc:1434] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260806 07:58:27.638162 6649 raft_consensus.cc:2272] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.639070 6531 tablet_bootstrap.cc:492] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.639230 6531 ts_tablet_manager.cc:1403] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.006s
I20260806 07:58:27.639606 6649 ts_tablet_manager.cc:1916] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.639827 6572 leader_election.cc:290] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.640621 6531 raft_consensus.cc:359] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.640810 6531 raft_consensus.cc:385] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.640918 6531 raft_consensus.cc:740] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.641225 6531 consensus_queue.cc:260] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.641722 6453 raft_consensus.cc:2468] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:27.642215 6250 leader_election.cc:304] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:27.642598 6538 raft_consensus.cc:2804] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.642740 6538 raft_consensus.cc:697] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:27.643030 6538 consensus_queue.cc:237] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.644586 6647 ts_tablet_manager.cc:1916] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.646289 6537 tablet_bootstrap.cc:492] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.646479 6537 ts_tablet_manager.cc:1403] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.003s
I20260806 07:58:27.646751 6532 tablet_bootstrap.cc:492] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.647306 6537 raft_consensus.cc:359] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.647535 6537 raft_consensus.cc:385] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.647631 6537 raft_consensus.cc:740] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.648135 6531 ts_tablet_manager.cc:1434] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260806 07:58:27.648180 6537 consensus_queue.cc:260] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.649147 6537 ts_tablet_manager.cc:1434] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:27.651244 6532 tablet_bootstrap.cc:654] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.651789 6537 tablet_bootstrap.cc:492] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.652421 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 0d04186bb02946928cc22152266d8a3f (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.653877 6537 tablet_bootstrap.cc:654] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.659792 6649 ts_tablet_manager.cc:1929] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.659997 6649 log.cc:1199] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/fb7a45d28865477bbe3cef826d477a93
I20260806 07:58:27.660425 6531 tablet_bootstrap.cc:492] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.660642 6649 ts_tablet_manager.cc:1950] T fb7a45d28865477bbe3cef826d477a93 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.662117 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet fb7a45d28865477bbe3cef826d477a93 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.662428 6648 tablet_replica.cc:333] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.662737 6648 raft_consensus.cc:2243] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.664199 6648 raft_consensus.cc:2272] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.666683 6531 tablet_bootstrap.cc:654] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.666276 6015 catalog_manager.cc:5697] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:27.666988 6647 ts_tablet_manager.cc:1929] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.667171 6647 log.cc:1199] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/fb7a45d28865477bbe3cef826d477a93
I20260806 07:58:27.667725 6647 ts_tablet_manager.cc:1950] T fb7a45d28865477bbe3cef826d477a93 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.667789 6649 tablet_replica.cc:333] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.667996 6649 raft_consensus.cc:2243] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.668227 6649 raft_consensus.cc:2272] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.669651 6647 tablet_replica.cc:333] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.669723 6649 ts_tablet_manager.cc:1916] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.669857 6647 raft_consensus.cc:2243] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.669888 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet fb7a45d28865477bbe3cef826d477a93 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.670202 6647 raft_consensus.cc:2272] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.671042 6647 ts_tablet_manager.cc:1916] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.678136 6537 tablet_bootstrap.cc:492] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.678280 6581 raft_consensus.cc:493] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.678319 6537 ts_tablet_manager.cc:1403] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.005s
I20260806 07:58:27.678427 6622 raft_consensus.cc:493] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.678475 6581 raft_consensus.cc:515] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.678694 6622 raft_consensus.cc:515] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.678998 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.679186 6581 leader_election.cc:290] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.679318 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.679548 6622 leader_election.cc:290] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.679615 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.679891 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.679919 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46f5138fc13a493bb7217f123071c20a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.680858 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46f5138fc13a493bb7217f123071c20a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.681083 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c0ddbb77d94c408ed054fd4dbdcf4e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.681361 6180 raft_consensus.cc:2468] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
W20260806 07:58:27.681512 6113 leader_election.cc:343] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.681769 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260806 07:58:27.681977 6114 leader_election.cc:343] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.682101 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.682113 6114 leader_election.cc:304] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:27.682318 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.682533 6581 raft_consensus.cc:2749] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:27.682540 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c0ddbb77d94c408ed054fd4dbdcf4e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.682708 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.682974 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.683240 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.683274 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.683662 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.683694 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.683944 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.683954 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.684247 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.685041 6537 raft_consensus.cc:359] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.685263 6537 raft_consensus.cc:385] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.685365 6537 raft_consensus.cc:740] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.685773 6537 consensus_queue.cc:260] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.686353 6537 ts_tablet_manager.cc:1434] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 07:58:27.686456 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.686671 6537 tablet_bootstrap.cc:492] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.686785 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.687052 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.688552 6537 tablet_bootstrap.cc:654] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:27.689010 6384 leader_election.cc:343] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.689514 6532 tablet_bootstrap.cc:492] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.689698 6532 ts_tablet_manager.cc:1403] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.007s
I20260806 07:58:27.690593 6532 raft_consensus.cc:359] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.690773 6532 raft_consensus.cc:385] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.690872 6532 raft_consensus.cc:740] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.691051 6647 ts_tablet_manager.cc:1929] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.691176 6647 log.cc:1199] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/523ed5cfaa914101a9946b4391aad3c0
I20260806 07:58:27.691144 6532 consensus_queue.cc:260] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.691692 6647 ts_tablet_manager.cc:1950] T 523ed5cfaa914101a9946b4391aad3c0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.691743 6532 ts_tablet_manager.cc:1434] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:27.692044 6532 tablet_bootstrap.cc:492] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.692791 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 523ed5cfaa914101a9946b4391aad3c0 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.693077 6538 raft_consensus.cc:493] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.693240 6538 raft_consensus.cc:515] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.693416 6647 tablet_replica.cc:333] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.693686 6647 raft_consensus.cc:2243] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.693842 6531 tablet_bootstrap.cc:492] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.694005 6531 ts_tablet_manager.cc:1403] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.013s
I20260806 07:58:27.694022 6647 raft_consensus.cc:2272] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.694911 6531 raft_consensus.cc:359] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.695187 6531 raft_consensus.cc:385] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.695273 6531 raft_consensus.cc:740] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.695587 6531 consensus_queue.cc:260] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.696292 6531 ts_tablet_manager.cc:1434] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:27.694018 6538 leader_election.cc:290] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.696691 6531 tablet_bootstrap.cc:492] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.697152 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0457d931c271465e8d3c17461af639a5" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.697396 6179 raft_consensus.cc:2468] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:27.697853 6252 leader_election.cc:304] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.698153 6538 raft_consensus.cc:2804] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.698238 6538 raft_consensus.cc:493] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.698347 6538 raft_consensus.cc:3060] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.699107 6531 tablet_bootstrap.cc:654] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.699304 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":168,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.700502 6581 raft_consensus.cc:493] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.689504 6387 leader_election.cc:304] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:27.700676 6581 raft_consensus.cc:515] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.701289 6568 raft_consensus.cc:2804] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.701395 6568 raft_consensus.cc:493] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.701409 6581 leader_election.cc:290] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.701514 6568 raft_consensus.cc:3060] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.702351 6649 ts_tablet_manager.cc:1929] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.702411 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cea29d556ed4c2cb5fc9a8e4197bc7b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.702513 6649 log.cc:1199] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/523ed5cfaa914101a9946b4391aad3c0
I20260806 07:58:27.702610 6317 raft_consensus.cc:2468] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:27.703079 6649 ts_tablet_manager.cc:1950] T 523ed5cfaa914101a9946b4391aad3c0 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.703063 6113 leader_election.cc:304] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.703378 6581 raft_consensus.cc:2804] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.703462 6581 raft_consensus.cc:493] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.703560 6581 raft_consensus.cc:3060] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.704195 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 523ed5cfaa914101a9946b4391aad3c0 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.704572 6568 raft_consensus.cc:515] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.705463 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0457d931c271465e8d3c17461af639a5" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.705549 6532 tablet_bootstrap.cc:654] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.705691 6453 raft_consensus.cc:2468] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:27.706566 6568 leader_election.cc:290] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.706687 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cea29d556ed4c2cb5fc9a8e4197bc7b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.706873 6453 raft_consensus.cc:2468] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:27.706938 6649 tablet_replica.cc:333] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.707162 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.707221 6649 raft_consensus.cc:2243] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.707314 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c0ddbb77d94c408ed054fd4dbdcf4e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:27.707441 6649 raft_consensus.cc:2272] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.707576 6179 raft_consensus.cc:3060] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.707584 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.707845 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.708187 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.708416 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.708496 6531 tablet_bootstrap.cc:492] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.708681 6531 ts_tablet_manager.cc:1403] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260806 07:58:27.709721 6531 raft_consensus.cc:359] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.709939 6531 raft_consensus.cc:385] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.710038 6531 raft_consensus.cc:740] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.710306 6649 ts_tablet_manager.cc:1916] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.710255 6531 consensus_queue.cc:260] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.710660 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c0ddbb77d94c408ed054fd4dbdcf4e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:27.711251 6384 leader_election.cc:343] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.711404 6531 ts_tablet_manager.cc:1434] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 07:58:27.711714 6531 tablet_bootstrap.cc:492] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.713490 6531 tablet_bootstrap.cc:654] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.717898 6179 raft_consensus.cc:2468] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:27.718389 6649 ts_tablet_manager.cc:1929] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.718554 6649 log.cc:1199] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/1bce3ad6757940269b58768139f10bbf
I20260806 07:58:27.719026 6649 ts_tablet_manager.cc:1950] T 1bce3ad6757940269b58768139f10bbf P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.720285 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 1bce3ad6757940269b58768139f10bbf (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.721051 6387 leader_election.cc:304] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:27.721688 6622 raft_consensus.cc:2804] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.721859 6622 raft_consensus.cc:697] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:27.722195 6622 consensus_queue.cc:237] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.719025 6581 raft_consensus.cc:515] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.725162 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cea29d556ed4c2cb5fc9a8e4197bc7b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:27.725415 6317 raft_consensus.cc:3060] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.727653 6649 tablet_replica.cc:333] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.727823 6544 raft_consensus.cc:493] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.727914 6649 raft_consensus.cc:2243] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:27.728003 6544 raft_consensus.cc:515] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.728327 6649 raft_consensus.cc:2272] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.728660 6544 leader_election.cc:290] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.729243 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":188,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.729815 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacb42bf87304fbe8aa1edf4cc6a9e58" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.730176 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.006s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":14281,"lbm_read_time_us":179,"lbm_reads_lt_1ms":8,"lbm_write_time_us":193,"lbm_writes_lt_1ms":4,"mutex_wait_us":9675,"reinsert_count":0,"update_count":2}
W20260806 07:58:27.730523 6113 leader_election.cc:343] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.732236 6649 ts_tablet_manager.cc:1916] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.732462 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.004s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":5464,"dirs.run_cpu_time_us":143,"dirs.run_wall_time_us":598,"lbm_read_time_us":167,"lbm_reads_lt_1ms":8,"lbm_write_time_us":216,"lbm_writes_lt_1ms":4,"mutex_wait_us":8390,"reinsert_count":0,"update_count":2}
I20260806 07:58:27.732892 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacb42bf87304fbe8aa1edf4cc6a9e58" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.733480 6317 raft_consensus.cc:2468] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
W20260806 07:58:27.733937 6114 leader_election.cc:343] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.734061 6648 ts_tablet_manager.cc:1916] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.734117 6114 leader_election.cc:304] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:27.734187 6538 raft_consensus.cc:515] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.734637 6544 raft_consensus.cc:2749] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:27.734122 6113 leader_election.cc:304] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.735297 6538 leader_election.cc:290] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.735510 6544 raft_consensus.cc:2804] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.735702 6544 raft_consensus.cc:697] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:27.735769 6581 leader_election.cc:290] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.737296 6544 consensus_queue.cc:237] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.739519 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0457d931c271465e8d3c17461af639a5" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:27.739813 6450 raft_consensus.cc:3060] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.741930 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cea29d556ed4c2cb5fc9a8e4197bc7b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:27.742177 6453 raft_consensus.cc:3060] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.742174 6647 ts_tablet_manager.cc:1916] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.743901 6648 ts_tablet_manager.cc:1929] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.744041 6648 log.cc:1199] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/8d71ed2aa2bf43ffbc98dda21fc73eac
I20260806 07:58:27.744510 6648 ts_tablet_manager.cc:1950] T 8d71ed2aa2bf43ffbc98dda21fc73eac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.745885 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 8d71ed2aa2bf43ffbc98dda21fc73eac (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.746567 6648 tablet_replica.cc:333] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.748317 6453 raft_consensus.cc:2468] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:27.749378 6450 raft_consensus.cc:2468] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:27.749446 6015 catalog_manager.cc:5697] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:27.749780 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0457d931c271465e8d3c17461af639a5" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:27.749996 6179 raft_consensus.cc:3060] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.752005 6537 tablet_bootstrap.cc:492] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.752175 6537 ts_tablet_manager.cc:1403] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.019s
I20260806 07:58:27.753002 6537 raft_consensus.cc:359] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.753232 6537 raft_consensus.cc:385] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.753306 6537 raft_consensus.cc:740] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.753545 6537 consensus_queue.cc:260] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.753693 6250 leader_election.cc:304] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:27.754019 6538 raft_consensus.cc:2804] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.754141 6537 ts_tablet_manager.cc:1434] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:27.754194 6538 raft_consensus.cc:697] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:27.754456 6537 tablet_bootstrap.cc:492] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.754475 6538 consensus_queue.cc:237] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.754619 6647 ts_tablet_manager.cc:1929] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.754796 6647 log.cc:1199] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/1bce3ad6757940269b58768139f10bbf
I20260806 07:58:27.755330 6647 ts_tablet_manager.cc:1950] T 1bce3ad6757940269b58768139f10bbf P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.756319 6537 tablet_bootstrap.cc:654] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.756772 6647 tablet_replica.cc:333] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.756999 6647 raft_consensus.cc:2243] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.757202 6647 raft_consensus.cc:2272] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.757493 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 1bce3ad6757940269b58768139f10bbf (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.758234 6647 ts_tablet_manager.cc:1916] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.758404 6531 tablet_bootstrap.cc:492] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.758620 6531 ts_tablet_manager.cc:1403] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.008s
I20260806 07:58:27.759764 6531 raft_consensus.cc:359] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.760004 6531 raft_consensus.cc:385] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.760082 6531 raft_consensus.cc:740] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.760412 6531 consensus_queue.cc:260] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.761196 6531 ts_tablet_manager.cc:1434] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:27.761574 6531 tablet_bootstrap.cc:492] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.763860 6531 tablet_bootstrap.cc:654] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.764915 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.765244 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.765523 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.765877 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.765938 6647 ts_tablet_manager.cc:1929] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.766093 6647 log.cc:1199] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/cbcae8fb644044d99a96dd9fb716965c
I20260806 07:58:27.766254 6179 raft_consensus.cc:2468] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:27.766448 6537 tablet_bootstrap.cc:492] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.766569 6647 ts_tablet_manager.cc:1950] T cbcae8fb644044d99a96dd9fb716965c P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.766608 6537 ts_tablet_manager.cc:1403] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.001s
I20260806 07:58:27.767560 6537 raft_consensus.cc:359] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.767800 6537 raft_consensus.cc:385] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.767903 6537 raft_consensus.cc:740] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.767891 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet cbcae8fb644044d99a96dd9fb716965c (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.768042 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.768195 6537 consensus_queue.cc:260] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.768349 6647 tablet_replica.cc:333] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:27.768391 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.768587 6647 raft_consensus.cc:2243] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.768642 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.768787 6647 raft_consensus.cc:2272] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.768862 6537 ts_tablet_manager.cc:1434] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:27.769163 6537 tablet_bootstrap.cc:492] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.769134 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.769564 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.769922 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.770218 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.770274 6647 ts_tablet_manager.cc:1916] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.771327 6532 tablet_bootstrap.cc:492] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.771503 6532 ts_tablet_manager.cc:1403] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.080s user 0.008s sys 0.007s
I20260806 07:58:27.771584 6537 tablet_bootstrap.cc:654] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.772302 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.772387 6532 raft_consensus.cc:359] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.772606 6532 raft_consensus.cc:385] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.772650 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.772699 6532 raft_consensus.cc:740] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.772902 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.776863 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.777173 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.777436 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.777895 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.778174 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.779141 6014 catalog_manager.cc:5697] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:27.780124 6647 ts_tablet_manager.cc:1929] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.780277 6647 log.cc:1199] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/b6d0ae6d095640eea8caa5853c1fbb2a
I20260806 07:58:27.780769 6647 ts_tablet_manager.cc:1950] T b6d0ae6d095640eea8caa5853c1fbb2a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:27.782260 6532 consensus_queue.cc:260] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.782996 6532 ts_tablet_manager.cc:1434] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260806 07:58:27.783344 6532 tablet_bootstrap.cc:492] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.785658 6532 tablet_bootstrap.cc:654] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.786710 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.787117 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.787485 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.787952 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.788139 6537 tablet_bootstrap.cc:492] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.788259 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.788349 6537 ts_tablet_manager.cc:1403] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.005s
I20260806 07:58:27.789346 6537 raft_consensus.cc:359] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.789158 6016 catalog_manager.cc:5697] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:27.789669 6537 raft_consensus.cc:385] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.789767 6537 raft_consensus.cc:740] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.790076 6537 consensus_queue.cc:260] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.790680 6537 ts_tablet_manager.cc:1434] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:27.791105 6537 tablet_bootstrap.cc:492] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.793325 6537 tablet_bootstrap.cc:654] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.794131 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet b6d0ae6d095640eea8caa5853c1fbb2a (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.795949 6532 tablet_bootstrap.cc:492] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.796118 6532 ts_tablet_manager.cc:1403] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260806 07:58:27.797035 6532 raft_consensus.cc:359] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.797261 6532 raft_consensus.cc:385] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.797363 6532 raft_consensus.cc:740] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.797700 6532 consensus_queue.cc:260] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.798357 6532 ts_tablet_manager.cc:1434] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:27.799266 6532 tablet_bootstrap.cc:492] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.801512 6532 tablet_bootstrap.cc:654] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.806468 6537 tablet_bootstrap.cc:492] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.806630 6537 ts_tablet_manager.cc:1403] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.001s
I20260806 07:58:27.807538 6537 raft_consensus.cc:359] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.807789 6537 raft_consensus.cc:385] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.807880 6537 raft_consensus.cc:740] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.808203 6537 consensus_queue.cc:260] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.809340 6649 ts_tablet_manager.cc:1929] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.809358 6544 raft_consensus.cc:493] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.809478 6649 log.cc:1199] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/cbcae8fb644044d99a96dd9fb716965c
I20260806 07:58:27.809505 6544 raft_consensus.cc:515] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.809957 6649 ts_tablet_manager.cc:1950] T cbcae8fb644044d99a96dd9fb716965c P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.810341 6544 leader_election.cc:290] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.811019 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89b495f1733475895ce958ae57861e1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.811124 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet cbcae8fb644044d99a96dd9fb716965c (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.811277 6317 raft_consensus.cc:2468] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:27.811908 6113 leader_election.cc:304] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.812058 6531 tablet_bootstrap.cc:492] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.812196 6531 ts_tablet_manager.cc:1403] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.026s
I20260806 07:58:27.812561 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89b495f1733475895ce958ae57861e1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
W20260806 07:58:27.813227 6114 leader_election.cc:343] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.813611 6544 raft_consensus.cc:2804] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.813712 6544 raft_consensus.cc:493] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.813822 6544 raft_consensus.cc:3060] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.817013 6544 raft_consensus.cc:515] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.817412 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":238,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.818157 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89b495f1733475895ce958ae57861e1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:27.818403 6317 raft_consensus.cc:3060] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.821570 6317 raft_consensus.cc:2468] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:27.822062 6113 leader_election.cc:304] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.822368 6583 raft_consensus.cc:2804] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.822558 6583 raft_consensus.cc:697] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:27.822844 6583 consensus_queue.cc:237] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.824333 6544 leader_election.cc:290] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.824318 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.824848 6537 ts_tablet_manager.cc:1434] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260806 07:58:27.825016 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89b495f1733475895ce958ae57861e1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:27.825187 6537 tablet_bootstrap.cc:492] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.824867 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260806 07:58:27.825906 6114 leader_election.cc:343] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.825964 6622 raft_consensus.cc:493] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.826167 6622 raft_consensus.cc:515] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.827170 6622 leader_election.cc:290] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.827261 6537 tablet_bootstrap.cc:654] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.827824 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.827921 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.828194 6179 raft_consensus.cc:2468] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
W20260806 07:58:27.828552 6384 leader_election.cc:343] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.828756 6387 leader_election.cc:304] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:27.829406 6622 raft_consensus.cc:2804] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.829568 6622 raft_consensus.cc:493] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.829876 6622 raft_consensus.cc:3060] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.835399 6622 raft_consensus.cc:515] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.838210 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:27.838577 6179 raft_consensus.cc:3060] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.839202 6531 raft_consensus.cc:359] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.839457 6531 raft_consensus.cc:385] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.839563 6531 raft_consensus.cc:740] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.839970 6531 consensus_queue.cc:260] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.840770 6531 ts_tablet_manager.cc:1434] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260806 07:58:27.841246 6531 tablet_bootstrap.cc:492] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.843590 6531 tablet_bootstrap.cc:654] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.845042 6649 tablet_replica.cc:333] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:27.845332 6649 raft_consensus.cc:2243] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.845577 6649 raft_consensus.cc:2272] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.846177 6179 raft_consensus.cc:2468] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:27.846740 6387 leader_election.cc:304] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.846827 6649 ts_tablet_manager.cc:1916] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.847224 6568 raft_consensus.cc:2804] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.847492 6568 raft_consensus.cc:697] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:27.847929 6568 consensus_queue.cc:237] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.854280 6544 raft_consensus.cc:493] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.854528 6531 tablet_bootstrap.cc:492] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:27.854476 6544 raft_consensus.cc:515] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.854699 6531 ts_tablet_manager.cc:1403] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260806 07:58:27.855520 6531 raft_consensus.cc:359] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.855747 6531 raft_consensus.cc:385] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.855795 6544 leader_election.cc:290] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.855839 6531 raft_consensus.cc:740] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.855798 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.856079 6531 consensus_queue.cc:260] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.856703 6531 ts_tablet_manager.cc:1434] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260806 07:58:27.856805 6113 leader_election.cc:343] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.857429 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.857802 6537 tablet_bootstrap.cc:492] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.857954 6537 ts_tablet_manager.cc:1403] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.017s
I20260806 07:58:27.858260 6114 leader_election.cc:304] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:27.858513 6531 tablet_bootstrap.cc:492] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.858770 6544 raft_consensus.cc:2749] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:58:27.859081 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.071s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":711,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":267,"lbm_reads_lt_1ms":8,"lbm_write_time_us":250,"lbm_writes_lt_1ms":4,"mutex_wait_us":30485,"reinsert_count":0,"update_count":3}
I20260806 07:58:27.859557 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.089s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":180,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.860147 6622 leader_election.cc:290] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.860394 6531 tablet_bootstrap.cc:654] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.860406 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.083s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":711,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":229,"lbm_reads_lt_1ms":8,"lbm_write_time_us":257,"lbm_writes_lt_1ms":4,"mutex_wait_us":15270,"reinsert_count":0,"update_count":3}
I20260806 07:58:27.746943 6648 raft_consensus.cc:2243] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.860951 6648 raft_consensus.cc:2272] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.860970 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:27.861935 6648 ts_tablet_manager.cc:1916] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 07:58:27.862353 6384 leader_election.cc:343] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.861935 6537 raft_consensus.cc:359] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.864333 6537 raft_consensus.cc:385] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.864404 6537 raft_consensus.cc:740] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.864656 6537 consensus_queue.cc:260] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.865185 6537 ts_tablet_manager.cc:1434] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 07:58:27.867664 6016 catalog_manager.cc:5697] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:27.870018 6648 ts_tablet_manager.cc:1929] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.870226 6648 log.cc:1199] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/fb7a45d28865477bbe3cef826d477a93
I20260806 07:58:27.870609 6537 tablet_bootstrap.cc:492] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.872386 6537 tablet_bootstrap.cc:654] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.874069 6648 ts_tablet_manager.cc:1950] T fb7a45d28865477bbe3cef826d477a93 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:27.875068 6648 tablet_replica.cc:333] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:27.875283 6648 raft_consensus.cc:2243] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:27.875407 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet fb7a45d28865477bbe3cef826d477a93 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.875489 6648 raft_consensus.cc:2272] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:27.836308 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.880293 6532 tablet_bootstrap.cc:492] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.880512 6532 ts_tablet_manager.cc:1403] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.081s user 0.020s sys 0.031s
I20260806 07:58:27.881471 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.171597
I20260806 07:58:27.881569 6532 raft_consensus.cc:359] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.881855 6532 raft_consensus.cc:385] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.881963 6532 raft_consensus.cc:740] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.882087 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:27.882357 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.886735 6532 consensus_queue.cc:260] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.887076 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.887683 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.889753 6532 ts_tablet_manager.cc:1434] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260806 07:58:27.890172 6532 tablet_bootstrap.cc:492] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.892457 6532 tablet_bootstrap.cc:654] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.898020 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.006s sys 0.000s Metrics: {"bytes_written":500,"delete_count":0,"lbm_write_time_us":194,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260806 07:58:27.853991 6015 catalog_manager.cc:5697] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:27.907289 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":19759}
I20260806 07:58:27.907585 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":19524}
I20260806 07:58:27.908788 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.107757
I20260806 07:58:27.913123 6622 raft_consensus.cc:493] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.913343 6622 raft_consensus.cc:515] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.913900 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:27.914175 6622 leader_election.cc:290] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.914171 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:27.915011 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ff8d074e9e4a7d9f788fb501528bae" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.915333 6317 raft_consensus.cc:2468] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:27.915884 6649 ts_tablet_manager.cc:1929] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:27.915947 6384 leader_election.cc:304] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:27.916044 6649 log.cc:1199] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/b6d0ae6d095640eea8caa5853c1fbb2a
I20260806 07:58:27.916599 6649 ts_tablet_manager.cc:1950] T b6d0ae6d095640eea8caa5853c1fbb2a P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:27.916734 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ff8d074e9e4a7d9f788fb501528bae" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.916955 6179 raft_consensus.cc:2468] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:27.917296 6622 raft_consensus.cc:2804] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.917389 6622 raft_consensus.cc:493] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.917493 6622 raft_consensus.cc:3060] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.917865 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet b6d0ae6d095640eea8caa5853c1fbb2a (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:27.920840 6622 raft_consensus.cc:515] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.922999 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ff8d074e9e4a7d9f788fb501528bae" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:27.923252 6179 raft_consensus.cc:3060] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.924780 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ff8d074e9e4a7d9f788fb501528bae" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:27.924990 6317 raft_consensus.cc:3060] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.925045 6532 tablet_bootstrap.cc:492] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.925226 6532 ts_tablet_manager.cc:1403] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.004s
I20260806 07:58:27.926430 6532 raft_consensus.cc:359] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.926683 6532 raft_consensus.cc:385] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.926813 6532 raft_consensus.cc:740] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.927177 6532 consensus_queue.cc:260] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.928196 6317 raft_consensus.cc:2468] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:27.928628 6384 leader_election.cc:304] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:27.929174 6568 raft_consensus.cc:2804] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.929320 6568 raft_consensus.cc:697] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:27.929302 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.105s user 0.007s sys 0.000s Metrics: {"bytes_written":472,"delete_count":0,"lbm_write_time_us":284,"lbm_writes_lt_1ms":4,"mutex_wait_us":39647,"reinsert_count":0,"update_count":2}
I20260806 07:58:27.929642 6568 consensus_queue.cc:237] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.907887 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.082s user 0.002s sys 0.005s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":175,"lbm_writes_lt_1ms":4,"mutex_wait_us":74390,"reinsert_count":0,"update_count":1}
I20260806 07:58:27.929956 6532 ts_tablet_manager.cc:1434] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 07:58:27.930320 6532 tablet_bootstrap.cc:492] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:27.932525 6532 tablet_bootstrap.cc:654] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.932790 6622 leader_election.cc:290] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:27.936717 6179 raft_consensus.cc:2468] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:27.941713 6537 tablet_bootstrap.cc:492] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:27.942339 6537 ts_tablet_manager.cc:1403] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.072s user 0.023s sys 0.019s
I20260806 07:58:27.943869 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.944278 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.944658 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.949266 6014 catalog_manager.cc:5697] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:27.949913 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.950534 6537 raft_consensus.cc:359] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.950821 6537 raft_consensus.cc:385] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.950953 6537 raft_consensus.cc:740] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.951294 6537 consensus_queue.cc:260] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.952468 6537 ts_tablet_manager.cc:1434] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260806 07:58:27.952816 6537 tablet_bootstrap.cc:492] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:27.954809 6537 tablet_bootstrap.cc:654] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:27.959734 6532 tablet_bootstrap.cc:492] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:27.959976 6532 ts_tablet_manager.cc:1403] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.002s
I20260806 07:58:27.961015 6532 raft_consensus.cc:359] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.961280 6532 raft_consensus.cc:385] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.961397 6532 raft_consensus.cc:740] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.962096 6538 raft_consensus.cc:493] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.962263 6538 raft_consensus.cc:515] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.963114 6538 leader_election.cc:290] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.963438 6544 raft_consensus.cc:493] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.963666 6544 raft_consensus.cc:515] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.964826 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacb42bf87304fbe8aa1edf4cc6a9e58" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.965123 6450 raft_consensus.cc:2468] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:27.964043 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacb42bf87304fbe8aa1edf4cc6a9e58" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:27.966346 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.967334 6250 leader_election.cc:304] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:27.968111 6179 raft_consensus.cc:2468] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:27.968581 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260806 07:58:27.968716 6113 leader_election.cc:343] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.969091 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:27.969565 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.964552 6544 leader_election.cc:290] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.970329 6648 ts_tablet_manager.cc:1916] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:27.970562 6538 raft_consensus.cc:2804] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.970697 6538 raft_consensus.cc:493] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.970804 6538 raft_consensus.cc:3060] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.972460 6317 raft_consensus.cc:3060] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.973553 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.092s user 0.004s sys 0.010s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":12507,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":32330,"dirs.run_cpu_time_us":366,"dirs.run_wall_time_us":2125,"lbm_read_time_us":728,"lbm_reads_lt_1ms":23,"lbm_write_time_us":413,"lbm_writes_lt_1ms":9,"mutex_wait_us":9119,"peak_mem_usage":32768,"reinsert_count":0,"update_count":6}
I20260806 07:58:27.974267 6544 raft_consensus.cc:493] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.974458 6544 raft_consensus.cc:515] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.975418 6531 tablet_bootstrap.cc:492] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
W20260806 07:58:27.975569 6114 leader_election.cc:343] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:27.975627 6531 ts_tablet_manager.cc:1403] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.117s user 0.005s sys 0.004s
I20260806 07:58:27.975780 6114 leader_election.cc:304] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:27.976361 6583 raft_consensus.cc:2749] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:27.976723 6531 raft_consensus.cc:359] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.976965 6531 raft_consensus.cc:385] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:27.977072 6531 raft_consensus.cc:740] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:27.977101 6544 leader_election.cc:290] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.977350 6531 consensus_queue.cc:260] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.978096 6531 ts_tablet_manager.cc:1434] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 07:58:27.978096 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8961173cf8d24b8eb022e994f90970da" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:27.978363 6318 raft_consensus.cc:2468] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:27.979142 6113 leader_election.cc:304] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:27.979883 6583 raft_consensus.cc:2804] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:27.981237 6538 raft_consensus.cc:515] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.982191 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8961173cf8d24b8eb022e994f90970da" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.982461 6450 raft_consensus.cc:2468] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:27.984692 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacb42bf87304fbe8aa1edf4cc6a9e58" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:27.984922 6450 raft_consensus.cc:3060] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.984939 6592 raft_consensus.cc:493] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.985144 6592 raft_consensus.cc:515] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } }
I20260806 07:58:27.985988 6592 leader_election.cc:290] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.988379 6450 raft_consensus.cc:2468] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:27.989071 6531 tablet_bootstrap.cc:492] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:27.989377 6250 leader_election.cc:304] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:27.989845 6592 raft_consensus.cc:2804] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:27.991782 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c0ddbb77d94c408ed054fd4dbdcf4e" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:27.992046 6538 leader_election.cc:290] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:27.992516 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":22839}
I20260806 07:58:27.992867 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":23095}
I20260806 07:58:27.993358 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.006s sys 0.000s Metrics: {"bytes_written":472,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"mutex_wait_us":16514,"reinsert_count":0,"update_count":2}
I20260806 07:58:27.993623 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":48410}
I20260806 07:58:27.994037 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":44435}
I20260806 07:58:27.994037 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.085s user 0.012s sys 0.004s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":12819,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":43254,"dirs.run_cpu_time_us":339,"dirs.run_wall_time_us":2260,"lbm_read_time_us":744,"lbm_reads_lt_1ms":29,"lbm_write_time_us":499,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":6}
I20260806 07:58:27.994932 6583 raft_consensus.cc:493] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:27.995174 6583 raft_consensus.cc:3060] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:27.961812 6532 consensus_queue.cc:260] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.996274 6568 consensus_queue.cc:1048] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:27.998359 6532 ts_tablet_manager.cc:1434] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.038s user 0.002s sys 0.002s
I20260806 07:58:27.998869 6572 raft_consensus.cc:493] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:27.999137 6572 raft_consensus.cc:515] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:27.999557 6568 consensus_queue.cc:1048] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.000448 6572 leader_election.cc:290] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.001829 6531 tablet_bootstrap.cc:654] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:28.001819 6113 consensus_peers.cc:597] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6d0ae6d095640eea8caa5853c1fbb2a. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.002133 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.057s user 0.007s sys 0.000s Metrics: {"bytes_written":472,"delete_count":0,"lbm_write_time_us":260,"lbm_writes_lt_1ms":4,"mutex_wait_us":26107,"reinsert_count":0,"update_count":2}
I20260806 07:58:28.002337 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacb42bf87304fbe8aa1edf4cc6a9e58" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:28.003165 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c0ddbb77d94c408ed054fd4dbdcf4e" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.003509 6178 raft_consensus.cc:2393] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:28.003937 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e093658a8714105a67c0165bc2978b1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.004268 6180 raft_consensus.cc:3060] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.004323 6252 leader_election.cc:304] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:28.004840 6572 raft_consensus.cc:3060] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.008656 6572 raft_consensus.cc:2749] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:28.003126 6583 raft_consensus.cc:515] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.008987 6180 raft_consensus.cc:2468] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:28.010819 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8961173cf8d24b8eb022e994f90970da" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.011150 6453 raft_consensus.cc:3060] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.015365 6531 tablet_bootstrap.cc:492] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.015614 6531 ts_tablet_manager.cc:1403] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.004s
I20260806 07:58:28.015810 6453 raft_consensus.cc:2468] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.016826 6531 raft_consensus.cc:359] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.017284 6531 raft_consensus.cc:385] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.017242 6114 leader_election.cc:304] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.017400 6531 raft_consensus.cc:740] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.017727 6544 raft_consensus.cc:2804] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.017814 6531 consensus_queue.cc:260] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.018071 6544 raft_consensus.cc:697] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:28.018468 6544 consensus_queue.cc:237] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.019590 6648 ts_tablet_manager.cc:1929] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.020099 6592 raft_consensus.cc:697] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:28.020545 6648 log.cc:1199] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/523ed5cfaa914101a9946b4391aad3c0
I20260806 07:58:28.020545 6592 consensus_queue.cc:237] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.021668 6648 ts_tablet_manager.cc:1950] T 523ed5cfaa914101a9946b4391aad3c0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.002910 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e093658a8714105a67c0165bc2978b1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.022886 6250 leader_election.cc:304] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:28.022192 6583 leader_election.cc:290] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.023766 6572 raft_consensus.cc:2749] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 07:58:28.024859 6531 ts_tablet_manager.cc:1434] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260806 07:58:28.026916 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 523ed5cfaa914101a9946b4391aad3c0 (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:28.027736 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8961173cf8d24b8eb022e994f90970da" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:28.027990 6317 raft_consensus.cc:3060] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.029393 6648 tablet_replica.cc:333] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.030009 6648 raft_consensus.cc:2243] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.030604 6648 raft_consensus.cc:2272] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.030718 6531 tablet_bootstrap.cc:492] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.031996 6648 ts_tablet_manager.cc:1916] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.031975 6317 raft_consensus.cc:2468] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.033131 6531 tablet_bootstrap.cc:654] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.033964 6016 catalog_manager.cc:5697] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.046543 6532 tablet_bootstrap.cc:492] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.049786 6532 tablet_bootstrap.cc:654] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.050375 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.050814 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.051159 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.051546 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.053814 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.054113 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.054418 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.054898 6531 tablet_bootstrap.cc:492] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.055071 6531 ts_tablet_manager.cc:1403] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20260806 07:58:28.055339 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.055644 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.055670 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.055965 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.055940 6531 raft_consensus.cc:359] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.056074 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.056152 6531 raft_consensus.cc:385] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.056244 6531 raft_consensus.cc:740] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.056366 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.056421 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.056473 6531 consensus_queue.cc:260] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.056653 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.056689 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.057126 6531 ts_tablet_manager.cc:1434] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:28.057411 6531 tablet_bootstrap.cc:492] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.057790 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.058068 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.058312 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.059283 6531 tablet_bootstrap.cc:654] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.060529 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.060847 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.061085 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.061415 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.061697 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.065008 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.002s sys 0.003s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":200,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.067575 6532 tablet_bootstrap.cc:492] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.067759 6532 ts_tablet_manager.cc:1403] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.004s
I20260806 07:58:28.068632 6532 raft_consensus.cc:359] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.068842 6532 raft_consensus.cc:385] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.068954 6532 raft_consensus.cc:740] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.069272 6532 consensus_queue.cc:260] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.069985 6532 ts_tablet_manager.cc:1434] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260806 07:58:28.070325 6532 tablet_bootstrap.cc:492] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.072278 6532 tablet_bootstrap.cc:654] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.077636 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.005s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":711,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":144,"lbm_reads_lt_1ms":8,"lbm_write_time_us":159,"lbm_writes_lt_1ms":4,"mutex_wait_us":5348,"reinsert_count":0,"update_count":3}
W20260806 07:58:28.043666 6114 consensus_peers.cc:597] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6d0ae6d095640eea8caa5853c1fbb2a. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.066506 6537 tablet_bootstrap.cc:492] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.080780 6537 ts_tablet_manager.cc:1403] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.128s user 0.030s sys 0.059s
I20260806 07:58:28.081740 6537 raft_consensus.cc:359] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.081960 6537 raft_consensus.cc:385] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.082038 6537 raft_consensus.cc:740] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.082298 6537 consensus_queue.cc:260] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.082877 6537 ts_tablet_manager.cc:1434] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:28.083250 6537 tablet_bootstrap.cc:492] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.085093 6537 tablet_bootstrap.cc:654] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.069726 6583 raft_consensus.cc:493] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.086123 6583 raft_consensus.cc:515] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.086863 6583 leader_election.cc:290] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.087443 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dd62212ec041f19f9760efb21ded44" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.087531 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dd62212ec041f19f9760efb21ded44" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.087711 6450 raft_consensus.cc:2468] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
W20260806 07:58:28.088303 6113 leader_election.cc:343] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:28.088517 6114 leader_election.cc:304] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:28.089033 6583 raft_consensus.cc:2804] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.089167 6583 raft_consensus.cc:493] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.089267 6583 raft_consensus.cc:3060] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.090286 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.029s user 0.002s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":711,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":3288,"lbm_read_time_us":207,"lbm_reads_lt_1ms":8,"lbm_write_time_us":200,"lbm_writes_lt_1ms":4,"mutex_wait_us":12345,"reinsert_count":0,"update_count":3}
I20260806 07:58:28.091737 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":203,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.094300 6623 raft_consensus.cc:493] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.094543 6623 raft_consensus.cc:515] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.095542 6623 leader_election.cc:290] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.096316 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46f5138fc13a493bb7217f123071c20a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.096520 6318 raft_consensus.cc:2468] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:28.096720 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46f5138fc13a493bb7217f123071c20a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.097012 6384 leader_election.cc:304] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:28.096995 6180 raft_consensus.cc:2468] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:28.097309 6623 raft_consensus.cc:2804] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.097426 6623 raft_consensus.cc:493] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.097509 6623 raft_consensus.cc:3060] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.100522 6623 raft_consensus.cc:515] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.100461 6016 catalog_manager.cc:5697] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.093281 6583 raft_consensus.cc:515] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.101809 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46f5138fc13a493bb7217f123071c20a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:28.101989 6180 raft_consensus.cc:3060] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.103231 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dd62212ec041f19f9760efb21ded44" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:28.104034 6113 leader_election.cc:343] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:28.104947 6583 leader_election.cc:290] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.105036 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dd62212ec041f19f9760efb21ded44" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.105134 6648 ts_tablet_manager.cc:1929] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.105245 6450 raft_consensus.cc:3060] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.105294 6648 log.cc:1199] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/1bce3ad6757940269b58768139f10bbf
I20260806 07:58:28.105832 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46f5138fc13a493bb7217f123071c20a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:28.106046 6318 raft_consensus.cc:3060] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.107523 6648 ts_tablet_manager.cc:1950] T 1bce3ad6757940269b58768139f10bbf P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.108433 6450 raft_consensus.cc:2468] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.108733 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 1bce3ad6757940269b58768139f10bbf (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:28.109058 6318 raft_consensus.cc:2468] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:28.109495 6384 leader_election.cc:304] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:28.109802 6568 raft_consensus.cc:2804] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.109972 6568 raft_consensus.cc:697] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:28.110287 6568 consensus_queue.cc:237] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.110595 6648 tablet_replica.cc:333] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.110844 6648 raft_consensus.cc:2243] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.111078 6648 raft_consensus.cc:2272] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.111533 6622 raft_consensus.cc:493] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.111696 6622 raft_consensus.cc:515] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.112599 6622 leader_election.cc:290] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.112807 6648 ts_tablet_manager.cc:1916] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.113554 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89b495f1733475895ce958ae57861e1" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.113677 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89b495f1733475895ce958ae57861e1" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.113947 6317 raft_consensus.cc:2393] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d4b117032604585af3544cb030a10f2 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:28.105147 6180 raft_consensus.cc:2468] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:28.115131 6384 leader_election.cc:304] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:28.115602 6622 raft_consensus.cc:3060] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.116014 6623 leader_election.cc:290] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.117913 6544 raft_consensus.cc:493] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.118086 6544 raft_consensus.cc:515] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.118817 6544 leader_election.cc:290] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.119491 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e540b33899435da8546e5080bfa40b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.119671 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.119635 6015 catalog_manager.cc:5697] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:28.120033 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.120349 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.120669 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.121491 6531 tablet_bootstrap.cc:492] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.121680 6531 ts_tablet_manager.cc:1403] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.064s user 0.008s sys 0.007s
I20260806 07:58:28.122614 6531 raft_consensus.cc:359] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.122699 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.122800 6531 raft_consensus.cc:385] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.122866 6531 raft_consensus.cc:740] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.123049 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.123229 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.123296 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.123577 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.123754 6532 tablet_bootstrap.cc:492] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.123904 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.123935 6532 ts_tablet_manager.cc:1403] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.054s user 0.002s sys 0.006s
I20260806 07:58:28.124202 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.124233 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.124539 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.124823 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.124765 6532 raft_consensus.cc:359] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.124961 6532 raft_consensus.cc:385] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.125048 6532 raft_consensus.cc:740] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.125176 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.125407 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.125474 6532 consensus_queue.cc:260] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.126082 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.126338 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.126616 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.126739 6532 ts_tablet_manager.cc:1434] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:28.127067 6532 tablet_bootstrap.cc:492] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.128484 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.128746 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.129004 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.128952 6114 leader_election.cc:304] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:28.129312 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.129558 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.129572 6583 raft_consensus.cc:2804] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.129760 6583 raft_consensus.cc:697] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:28.130095 6583 consensus_queue.cc:237] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.130216 6532 tablet_bootstrap.cc:654] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.133293 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.001s sys 0.004s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":206,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.139951 6532 tablet_bootstrap.cc:492] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.140128 6532 ts_tablet_manager.cc:1403] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260806 07:58:28.141201 6532 raft_consensus.cc:359] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.141397 6532 raft_consensus.cc:385] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.141497 6532 raft_consensus.cc:740] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.141773 6532 consensus_queue.cc:260] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.143790 6014 catalog_manager.cc:5697] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.143836 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.006s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":739,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":56,"lbm_read_time_us":175,"lbm_reads_lt_1ms":8,"lbm_write_time_us":168,"lbm_writes_lt_1ms":4,"mutex_wait_us":8180,"reinsert_count":0,"update_count":4}
I20260806 07:58:28.144438 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.144726 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.144970 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.145179 6532 ts_tablet_manager.cc:1434] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 07:58:28.145290 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.145900 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.146268 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.147260 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.147502 6532 tablet_bootstrap.cc:492] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.147497 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.147770 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.149326 6532 tablet_bootstrap.cc:654] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.152141 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.152778 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.153234 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.153631 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.153721 6583 consensus_queue.cc:1048] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.157472 6532 tablet_bootstrap.cc:492] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.157663 6532 ts_tablet_manager.cc:1403] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.008s
I20260806 07:58:28.158274 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.158787 6532 raft_consensus.cc:359] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.159101 6532 raft_consensus.cc:385] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.159222 6532 raft_consensus.cc:740] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.159502 6532 consensus_queue.cc:260] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.160003 6532 ts_tablet_manager.cc:1434] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:28.160468 6532 tablet_bootstrap.cc:492] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.162391 6532 tablet_bootstrap.cc:654] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.162896 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.163461 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.164954 6532 tablet_bootstrap.cc:492] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.165118 6532 ts_tablet_manager.cc:1403] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 07:58:28.166008 6532 raft_consensus.cc:359] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.166178 6532 raft_consensus.cc:385] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.166270 6532 raft_consensus.cc:740] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.166530 6532 consensus_queue.cc:260] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.167762 6532 ts_tablet_manager.cc:1434] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:28.168053 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.004s sys 0.003s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":739,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1034,"dirs.run_cpu_time_us":151,"dirs.run_wall_time_us":1664,"lbm_read_time_us":162,"lbm_reads_lt_1ms":8,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"mutex_wait_us":26694,"reinsert_count":0,"update_count":4}
I20260806 07:58:28.168802 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.169363
I20260806 07:58:28.169090 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.169153 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":22521}
I20260806 07:58:28.169443 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.169761 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.170004 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.170609 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.005s sys 0.000s Metrics: {"bytes_written":472,"delete_count":0,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"mutex_wait_us":17502,"reinsert_count":0,"update_count":2}
I20260806 07:58:28.173760 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":164,"lbm_writes_lt_1ms":4,"mutex_wait_us":7115,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.173843 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":12946}
I20260806 07:58:28.174297 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":10880}
I20260806 07:58:28.174885 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.003s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":762,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":45,"dirs.run_cpu_time_us":140,"dirs.run_wall_time_us":778,"lbm_read_time_us":134,"lbm_reads_lt_1ms":4,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":5}
I20260806 07:58:28.175395 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.175701 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.175976 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.176540 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.176476 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.003s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":793,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":47,"lbm_read_time_us":233,"lbm_reads_lt_1ms":12,"lbm_write_time_us":144,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":6}
I20260806 07:58:28.176900 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.176963 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":12997}
I20260806 07:58:28.177642 6568 consensus_queue.cc:1048] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260806 07:58:28.178186 6537 tablet_bootstrap.cc:492] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.178381 6537 ts_tablet_manager.cc:1403] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.095s user 0.008s sys 0.011s
I20260806 07:58:28.179598 6531 consensus_queue.cc:260] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.180112 6622 raft_consensus.cc:2749] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:28.180353 6531 ts_tablet_manager.cc:1434] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.059s user 0.002s sys 0.000s
I20260806 07:58:28.180835 6531 tablet_bootstrap.cc:492] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.183645 6648 ts_tablet_manager.cc:1929] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.184767 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.004s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":791,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":44,"lbm_read_time_us":249,"lbm_reads_lt_1ms":12,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":6}
I20260806 07:58:28.181524 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e540b33899435da8546e5080bfa40b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.186030 6648 log.cc:1199] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/cbcae8fb644044d99a96dd9fb716965c
I20260806 07:58:28.186518 6648 ts_tablet_manager.cc:1950] T cbcae8fb644044d99a96dd9fb716965c P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.187234 6572 raft_consensus.cc:493] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.187417 6572 raft_consensus.cc:515] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.187436 6648 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39230 (request call id 30) took 1058 ms. Trace:
I20260806 07:58:28.188236 6572 leader_election.cc:290] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.188412 6572 consensus_queue.cc:1048] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.187647 6648 rpcz_store.cc:276] 0806 07:58:27.129219 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:27.129271 (+ 52us) service_pool.cc:224] Handling call
0806 07:58:28.110480 (+981209us) ts_tablet_manager.cc:1151] Deleting tablet cbcae8fb644044d99a96dd9fb716965c
0806 07:58:28.110492 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:28.178299 (+ 67807us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:28.183549 (+ 5250us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:28.187398 (+ 3849us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":981005}
I20260806 07:58:28.188925 6648 tablet_replica.cc:333] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.189211 6648 raft_consensus.cc:2243] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.189180 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet cbcae8fb644044d99a96dd9fb716965c (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:28.189574 6648 raft_consensus.cc:2272] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.190706 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bafa7287f5b4914a9fa73cc10ac5d2b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.190965 6180 raft_consensus.cc:2468] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:28.191254 6450 raft_consensus.cc:2393] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in current term 1: Already voted for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in this term.
I20260806 07:58:28.191416 6252 leader_election.cc:304] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.191784 6592 raft_consensus.cc:2804] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.191907 6592 raft_consensus.cc:493] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.192006 6592 raft_consensus.cc:3060] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.195122 6537 raft_consensus.cc:359] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.195345 6537 raft_consensus.cc:385] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.195451 6537 raft_consensus.cc:740] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.195705 6537 consensus_queue.cc:260] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.195783 6583 consensus_queue.cc:1048] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.196173 6531 tablet_bootstrap.cc:654] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.196287 6537 ts_tablet_manager.cc:1434] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260806 07:58:28.197655 6648 ts_tablet_manager.cc:1916] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.203534 6538 consensus_queue.cc:1048] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.204772 6648 ts_tablet_manager.cc:1929] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.204917 6648 log.cc:1199] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/b6d0ae6d095640eea8caa5853c1fbb2a
I20260806 07:58:28.205392 6648 ts_tablet_manager.cc:1950] T b6d0ae6d095640eea8caa5853c1fbb2a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.206290 6648 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39230 (request call id 31) took 1075 ms. Trace:
I20260806 07:58:28.206398 6648 rpcz_store.cc:276] 0806 07:58:27.130381 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:27.130429 (+ 48us) service_pool.cc:224] Handling call
0806 07:58:28.188848 (+1058419us) ts_tablet_manager.cc:1151] Deleting tablet b6d0ae6d095640eea8caa5853c1fbb2a
0806 07:58:28.188858 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:28.201206 (+ 12348us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:28.204699 (+ 3493us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:28.206231 (+ 1532us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1058228}
I20260806 07:58:28.206861 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet b6d0ae6d095640eea8caa5853c1fbb2a (table table-0 [id=b25d41ac7e8c429383f4a49f46f6bb39]) successfully deleted
I20260806 07:58:28.208284 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bafa7287f5b4914a9fa73cc10ac5d2b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.209753 6623 consensus_queue.cc:1048] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260806 07:58:28.210088 6592 raft_consensus.cc:515] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.210368 6453 raft_consensus.cc:2468] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:28.210780 6592 leader_election.cc:290] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.211953 6114 leader_election.cc:304] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:28.212205 6583 raft_consensus.cc:493] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.212359 6583 raft_consensus.cc:515] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.212987 6583 leader_election.cc:290] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.213146 6583 raft_consensus.cc:3060] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.213583 6316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee24a9ece7f461a96e7f386b89c6208" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.213898 6316 raft_consensus.cc:2468] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.214527 6113 leader_election.cc:304] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.214949 6581 raft_consensus.cc:2804] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.215099 6581 raft_consensus.cc:493] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.215215 6581 raft_consensus.cc:3060] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.216594 6583 raft_consensus.cc:2749] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:28.221334 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bafa7287f5b4914a9fa73cc10ac5d2b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:28.222609 6179 raft_consensus.cc:3060] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.224033 6449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee24a9ece7f461a96e7f386b89c6208" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.224341 6449 raft_consensus.cc:2468] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.225919 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bafa7287f5b4914a9fa73cc10ac5d2b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.226147 6453 raft_consensus.cc:3060] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.227037 6581 raft_consensus.cc:515] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.227552 6718 consensus_queue.cc:1048] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.227844 6581 leader_election.cc:290] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.229017 6453 raft_consensus.cc:2468] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:28.229259 6449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee24a9ece7f461a96e7f386b89c6208" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.229434 6449 raft_consensus.cc:3060] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.232451 6449 raft_consensus.cc:2468] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.232999 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee24a9ece7f461a96e7f386b89c6208" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:28.233237 6317 raft_consensus.cc:3060] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.236152 6317 raft_consensus.cc:2468] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.236662 6113 leader_election.cc:304] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.237074 6583 raft_consensus.cc:2804] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.237223 6583 raft_consensus.cc:697] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:28.237540 6583 consensus_queue.cc:237] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.237695 6179 raft_consensus.cc:2468] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:28.238472 6250 leader_election.cc:304] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:28.238992 6538 raft_consensus.cc:2804] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.239197 6538 raft_consensus.cc:697] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:28.239253 6718 consensus_queue.cc:1048] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.239478 6538 consensus_queue.cc:237] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.239691 6718 raft_consensus.cc:493] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.239813 6718 raft_consensus.cc:515] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.240451 6718 leader_election.cc:290] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.241514 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.242256 6179 raft_consensus.cc:2468] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:28.243862 6252 leader_election.cc:304] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.244299 6572 raft_consensus.cc:2804] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.244439 6572 raft_consensus.cc:493] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.244549 6572 raft_consensus.cc:3060] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.249874 6572 raft_consensus.cc:515] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.252842 6572 leader_election.cc:290] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.254485 6449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
W20260806 07:58:28.256101 6250 leader_election.cc:343] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:28.256738 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:28.259585 6016 catalog_manager.cc:5697] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.264139 6592 raft_consensus.cc:493] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.264314 6592 raft_consensus.cc:515] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.265014 6592 leader_election.cc:290] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.264876 6015 catalog_manager.cc:5697] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.265931 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.266155 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
W20260806 07:58:28.267061 6250 leader_election.cc:343] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:28.267812 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cd956622e64e7fbfd27f370d63605b" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.270524 6180 raft_consensus.cc:2393] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:28.271183 6252 leader_election.cc:304] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:28.271733 6538 raft_consensus.cc:3060] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.274643 6538 raft_consensus.cc:2749] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:28.274883 6177 raft_consensus.cc:3060] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.276428 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.276767 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.277091 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.277451 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.277459 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.277953 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.278330 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.278324 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.278611 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.278643 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.278873 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.278950 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.279181 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.279443 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.281239 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.281358 6177 raft_consensus.cc:2468] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:28.281613 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.281905 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.281884 6252 leader_election.cc:304] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 5d4b117032604585af3544cb030a10f2
I20260806 07:58:28.282301 6592 raft_consensus.cc:2804] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.282480 6592 raft_consensus.cc:697] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:28.282771 6592 consensus_queue.cc:237] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.284068 6583 raft_consensus.cc:493] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.284227 6583 raft_consensus.cc:515] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.284896 6583 leader_election.cc:290] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.285434 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.285943 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.286024 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "462655eb372b4284a2b88378bbb0f686" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.286218 6317 raft_consensus.cc:2468] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.286693 6113 leader_election.cc:304] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.287043 6583 raft_consensus.cc:2804] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.287137 6583 raft_consensus.cc:493] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.287250 6583 raft_consensus.cc:3060] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.290205 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.290496 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.291280 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "462655eb372b4284a2b88378bbb0f686" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.291530 6450 raft_consensus.cc:2468] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.292084 6583 raft_consensus.cc:515] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.292817 6583 leader_election.cc:290] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.292980 6531 tablet_bootstrap.cc:492] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.293159 6531 ts_tablet_manager.cc:1403] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.113s user 0.004s sys 0.003s
I20260806 07:58:28.293659 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "462655eb372b4284a2b88378bbb0f686" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.293891 6450 raft_consensus.cc:3060] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.293928 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.294174 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.294440 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.294924 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.295171 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.295986 6531 raft_consensus.cc:359] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.296284 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "462655eb372b4284a2b88378bbb0f686" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:28.296550 6317 raft_consensus.cc:3060] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.304481 6317 raft_consensus.cc:2468] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.304781 6531 raft_consensus.cc:385] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.304986 6531 raft_consensus.cc:740] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.305114 6113 leader_election.cc:304] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.305689 6583 raft_consensus.cc:2804] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.305675 6531 consensus_queue.cc:260] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.306717 6531 ts_tablet_manager.cc:1434] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260806 07:58:28.308204 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.005s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":193,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":27008,"update_count":1}
I20260806 07:58:28.309033 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.309545 6583 raft_consensus.cc:697] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:28.309894 6583 consensus_queue.cc:237] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.311990 6015 catalog_manager.cc:5697] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.313757 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.035s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":820,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1307,"dirs.run_cpu_time_us":140,"dirs.run_wall_time_us":800,"lbm_read_time_us":255,"lbm_reads_lt_1ms":8,"lbm_write_time_us":216,"lbm_writes_lt_1ms":4,"mutex_wait_us":18822,"reinsert_count":0,"update_count":7}
I20260806 07:58:28.296761 6450 raft_consensus.cc:2468] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.314872 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":23952}
I20260806 07:58:28.315418 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.003s sys 0.003s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.315653 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.316063 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.319614 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.315106 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":154,"lbm_writes_lt_1ms":4,"mutex_wait_us":18845,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.319955 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.315311 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":5894}
I20260806 07:58:28.322201 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.006s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":820,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":7326,"lbm_read_time_us":168,"lbm_reads_lt_1ms":8,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"mutex_wait_us":7860,"reinsert_count":0,"update_count":7}
I20260806 07:58:28.322815 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101603
I20260806 07:58:28.323165 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.323585 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.101603
I20260806 07:58:28.326249 6014 catalog_manager.cc:5697] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.327618 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.328012 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.330399 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":845,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":195,"lbm_read_time_us":219,"lbm_reads_lt_1ms":12,"lbm_write_time_us":229,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":8}
I20260806 07:58:28.330987 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.331370 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.331673 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.333921 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.337137 6622 raft_consensus.cc:493] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.337473 6622 raft_consensus.cc:515] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.338615 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":144,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.339109 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.135817
I20260806 07:58:28.339213 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.339399 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.339515 6177 raft_consensus.cc:2393] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d4b117032604585af3544cb030a10f2 in current term 1: Already voted for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in this term.
I20260806 07:58:28.339783 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.135817
I20260806 07:58:28.341079 6622 leader_election.cc:290] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.344776 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.001s sys 0.002s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":385,"lbm_writes_lt_1ms":4,"mutex_wait_us":5231,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.345381 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.345614 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.002s sys 0.002s Metrics: {"bytes_written":472,"delete_count":0,"lbm_write_time_us":139,"lbm_writes_lt_1ms":4,"mutex_wait_us":38951,"reinsert_count":0,"update_count":2}
I20260806 07:58:28.345927 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"mutex_wait_us":25358,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.347255 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":13675}
I20260806 07:58:28.347743 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cc4788571140b2ab78811966d2e06d" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.350663 6014 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-1" } modify_external_catalogs: true
I20260806 07:58:28.350860 6014 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-1" } modify_external_catalogs: true
I20260806 07:58:28.351580 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.011s sys 0.001s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":13132,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":110,"lbm_read_time_us":1302,"lbm_reads_lt_1ms":29,"lbm_write_time_us":280,"lbm_writes_lt_1ms":9,"mutex_wait_us":15191,"peak_mem_usage":32768,"reinsert_count":0,"update_count":10}
I20260806 07:58:28.353943 6384 leader_election.cc:304] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:28.354725 6622 raft_consensus.cc:3060] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.357564 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.027s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":27321}
I20260806 07:58:28.357862 6622 raft_consensus.cc:2749] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:28.362140 6583 consensus_queue.cc:1048] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.367722 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.012s sys 0.004s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":13121,"cfile_init":6,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":615,"dirs.run_cpu_time_us":205,"dirs.run_wall_time_us":1225,"lbm_read_time_us":611,"lbm_reads_lt_1ms":33,"lbm_write_time_us":2520,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"mutex_wait_us":8072,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":3976,"threads_started":1,"update_count":10}
I20260806 07:58:28.368397 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.368698 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.368973 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.369247 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.369478 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.369751 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.369936 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.369989 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.370277 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.370585 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.370976 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.371256 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.372989 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet e1c0ddbb77d94c408ed054fd4dbdcf4e
I20260806 07:58:28.373685 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 4e093658a8714105a67c0165bc2978b1
I20260806 07:58:28.374084 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 0457d931c271465e8d3c17461af639a5
I20260806 07:58:28.374488 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 46f5138fc13a493bb7217f123071c20a
I20260806 07:58:28.374933 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 88ff8d074e9e4a7d9f788fb501528bae
I20260806 07:58:28.375332 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b
I20260806 07:58:28.375787 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet c0e540b33899435da8546e5080bfa40b
I20260806 07:58:28.376151 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet f89b495f1733475895ce958ae57861e1
I20260806 07:58:28.376595 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 59cd956622e64e7fbfd27f370d63605b
I20260806 07:58:28.376803 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.002s Metrics: {"bytes_written":929,"delete_count":0,"lbm_write_time_us":169,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:28.377067 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet aacb42bf87304fbe8aa1edf4cc6a9e58
I20260806 07:58:28.378249 6146 tablet_service.cc:1558] Processing DeleteTablet for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.378574 6648 tablet_replica.cc:333] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.378804 6648 raft_consensus.cc:2243] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.379053 6648 raft_consensus.cc:2272] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.379359 6156 tablet_service.cc:1558] Processing DeleteTablet for tablet c0e540b33899435da8546e5080bfa40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.380219 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e093658a8714105a67c0165bc2978b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.380270 6146 tablet_service.cc:1558] Processing DeleteTablet for tablet 88ff8d074e9e4a7d9f788fb501528bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.380489 6149 tablet_service.cc:1558] Processing DeleteTablet for tablet 0457d931c271465e8d3c17461af639a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.380731 6160 tablet_service.cc:1558] Processing DeleteTablet for tablet 46f5138fc13a493bb7217f123071c20a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.380936 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.381138 6152 tablet_service.cc:1558] Processing DeleteTablet for tablet f89b495f1733475895ce958ae57861e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.381387 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 59cd956622e64e7fbfd27f370d63605b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.381632 6153 tablet_service.cc:1558] Processing DeleteTablet for tablet aacb42bf87304fbe8aa1edf4cc6a9e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.381906 6294 tablet_service.cc:1558] Processing DeleteTablet for tablet f89b495f1733475895ce958ae57861e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.382094 6286 tablet_service.cc:1558] Processing DeleteTablet for tablet 59cd956622e64e7fbfd27f370d63605b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.382316 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet c0e540b33899435da8546e5080bfa40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.377563 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 8961173cf8d24b8eb022e994f90970da
I20260806 07:58:28.382545 6296 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.382788 6293 tablet_service.cc:1558] Processing DeleteTablet for tablet 88ff8d074e9e4a7d9f788fb501528bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.383034 6288 tablet_service.cc:1558] Processing DeleteTablet for tablet 46f5138fc13a493bb7217f123071c20a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.383257 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet 0457d931c271465e8d3c17461af639a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.383273 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 7bafa7287f5b4914a9fa73cc10ac5d2b
I20260806 07:58:28.383499 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e093658a8714105a67c0165bc2978b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.383726 6284 tablet_service.cc:1558] Processing DeleteTablet for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.383896 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 0ee24a9ece7f461a96e7f386b89c6208
I20260806 07:58:28.383920 6432 tablet_service.cc:1558] Processing DeleteTablet for tablet e1c0ddbb77d94c408ed054fd4dbdcf4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.384308 6649 tablet_replica.cc:333] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.384487 6424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0457d931c271465e8d3c17461af639a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.384557 6649 raft_consensus.cc:2243] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.384711 6433 tablet_service.cc:1558] Processing DeleteTablet for tablet 46f5138fc13a493bb7217f123071c20a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.384723 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 462655eb372b4284a2b88378bbb0f686
I20260806 07:58:28.384862 6648 ts_tablet_manager.cc:1916] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.384922 6429 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.384999 6649 raft_consensus.cc:2272] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.385142 6421 tablet_service.cc:1558] Processing DeleteTablet for tablet c0e540b33899435da8546e5080bfa40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.385355 6427 tablet_service.cc:1558] Processing DeleteTablet for tablet f89b495f1733475895ce958ae57861e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.385541 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 59cd956622e64e7fbfd27f370d63605b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.385753 6425 tablet_service.cc:1558] Processing DeleteTablet for tablet aacb42bf87304fbe8aa1edf4cc6a9e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.385937 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 8961173cf8d24b8eb022e994f90970da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.386173 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 06cc4788571140b2ab78811966d2e06d
I20260806 07:58:28.386336 6295 tablet_service.cc:1558] Processing DeleteTablet for tablet aacb42bf87304fbe8aa1edf4cc6a9e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.383965 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 88ff8d074e9e4a7d9f788fb501528bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.384279 6423 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e093658a8714105a67c0165bc2978b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.386778 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet 462655eb372b4284a2b88378bbb0f686 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.386987 6152 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.387167 6160 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cc4788571140b2ab78811966d2e06d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.387218 6649 ts_tablet_manager.cc:1916] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.387274 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 81dd62212ec041f19f9760efb21ded44
I20260806 07:58:28.387352 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.387570 6153 tablet_service.cc:1558] Processing DeleteTablet for tablet 8961173cf8d24b8eb022e994f90970da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.390443 6014 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46090:
name: "table-2"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260806 07:58:28.391033 6425 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cc4788571140b2ab78811966d2e06d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.391229 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 462655eb372b4284a2b88378bbb0f686 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.391412 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 81dd62212ec041f19f9760efb21ded44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.391607 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
W20260806 07:58:28.391646 6014 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 07:58:28.391783 6423 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:28.392135 6153 tablet_service.cc:1558] Processing DeleteTablet for tablet 81dd62212ec041f19f9760efb21ded44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:28.393326 6284 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bafa7287f5b4914a9fa73cc10ac5d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.393559 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet 462655eb372b4284a2b88378bbb0f686 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.393787 6293 tablet_service.cc:1558] Processing DeleteTablet for tablet 81dd62212ec041f19f9760efb21ded44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.393961 6295 tablet_service.cc:1558] Processing DeleteTablet for tablet 8961173cf8d24b8eb022e994f90970da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.394133 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.394301 6288 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cc4788571140b2ab78811966d2e06d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:28.396396 6755 tablet_replica.cc:333] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.396646 6755 raft_consensus.cc:2243] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.396847 6755 pending_rounds.cc:70] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Trying to abort 1 pending ops.
I20260806 07:58:28.397050 6755 pending_rounds.cc:77] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468729644830720 op_type: NO_OP noop_request { }
I20260806 07:58:28.397373 6755 raft_consensus.cc:2889] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:28.397524 6755 raft_consensus.cc:2272] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.398461 6755 ts_tablet_manager.cc:1916] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 07:58:28.399655 6113 consensus_peers.cc:597] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.401360 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.032s user 0.006s sys 0.000s Metrics: {"bytes_written":929,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:28.403097 6622 consensus_queue.cc:1048] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.403792 6648 ts_tablet_manager.cc:1929] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.403934 6648 log.cc:1199] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/e1c0ddbb77d94c408ed054fd4dbdcf4e
I20260806 07:58:28.404448 6648 ts_tablet_manager.cc:1950] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.404733 6623 consensus_queue.cc:1048] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.405911 6755 ts_tablet_manager.cc:1929] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.405939 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet e1c0ddbb77d94c408ed054fd4dbdcf4e (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.406104 6755 log.cc:1199] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/f89b495f1733475895ce958ae57861e1
I20260806 07:58:28.406515 6755 ts_tablet_manager.cc:1950] T f89b495f1733475895ce958ae57861e1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.406580 6648 tablet_replica.cc:333] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.406809 6648 raft_consensus.cc:2243] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.407014 6648 raft_consensus.cc:2272] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.407611 6755 tablet_replica.cc:333] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.407819 6755 raft_consensus.cc:2243] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.408043 6755 raft_consensus.cc:2272] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.408686 6581 consensus_queue.cc:1048] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 07:58:28.410553 6384 consensus_peers.cc:597] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.411370 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet f89b495f1733475895ce958ae57861e1 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.416290 6649 ts_tablet_manager.cc:1929] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.419651 6649 log.cc:1199] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/e1c0ddbb77d94c408ed054fd4dbdcf4e
I20260806 07:58:28.420342 6649 ts_tablet_manager.cc:1950] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.422178 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.422833 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.423368 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet e1c0ddbb77d94c408ed054fd4dbdcf4e (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.430392 6648 ts_tablet_manager.cc:1916] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.432909 6592 raft_consensus.cc:493] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.433111 6592 raft_consensus.cc:515] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.432996 6649 tablet_replica.cc:333] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.433648 6649 raft_consensus.cc:2243] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.433923 6592 leader_election.cc:290] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.434266 6649 raft_consensus.cc:2272] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.435081 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dd62212ec041f19f9760efb21ded44" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.435036 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dd62212ec041f19f9760efb21ded44" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.435570 6451 raft_consensus.cc:2393] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:28.436262 6250 leader_election.cc:304] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:28.436712 6592 raft_consensus.cc:3060] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.439684 6592 raft_consensus.cc:2749] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:28.440536 6649 ts_tablet_manager.cc:1916] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.440788 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.012s sys 0.002s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":863,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":5291,"drs_written":1,"lbm_read_time_us":124,"lbm_reads_lt_1ms":4,"lbm_write_time_us":974,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260806 07:58:28.441736 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:28.442075 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:28.442404 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259786
I20260806 07:58:28.442777 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.442975 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.259786
I20260806 07:58:28.442411 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":88,"lbm_reads_lt_1ms":4}
I20260806 07:58:28.444211 6592 consensus_queue.cc:1048] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.445946 6755 ts_tablet_manager.cc:1916] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.461705 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.462074 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.463342 6622 consensus_queue.cc:1048] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.470839 6648 ts_tablet_manager.cc:1929] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.471055 6648 log.cc:1199] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/c0e540b33899435da8546e5080bfa40b
I20260806 07:58:28.471508 6002 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:28.471585 6648 ts_tablet_manager.cc:1950] T c0e540b33899435da8546e5080bfa40b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.473042 6649 ts_tablet_manager.cc:1929] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.473205 6649 log.cc:1199] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/0457d931c271465e8d3c17461af639a5
I20260806 07:58:28.476272 6592 consensus_queue.cc:1048] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.478335 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.005s sys 0.000s Metrics: {"bytes_written":464,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":3}
I20260806 07:58:28.478909 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.480098 6755 ts_tablet_manager.cc:1929] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:28.480312 6755 log.cc:1199] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/59cd956622e64e7fbfd27f370d63605b
I20260806 07:58:28.480805 6755 ts_tablet_manager.cc:1950] T 59cd956622e64e7fbfd27f370d63605b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.481926 6755 tablet_replica.cc:333] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.482045 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 59cd956622e64e7fbfd27f370d63605b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.482296 6755 raft_consensus.cc:2243] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.482681 6755 raft_consensus.cc:2272] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.483714 6755 ts_tablet_manager.cc:1916] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.488184 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.007s sys 0.000s Metrics: {"dirs.queue_time_us":8601,"dirs.run_cpu_time_us":878,"dirs.run_wall_time_us":7434,"lbm_write_time_us":359,"lbm_writes_lt_1ms":7,"mutex_wait_us":12723,"num_input_rowsets":0,"thread_start_us":9165,"threads_started":1}
I20260806 07:58:28.488847 6648 tablet_replica.cc:333] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.488879 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet c0e540b33899435da8546e5080bfa40b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.489126 6648 raft_consensus.cc:2243] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.489384 6648 raft_consensus.cc:2272] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.489434 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.489627 6649 ts_tablet_manager.cc:1950] T 0457d931c271465e8d3c17461af639a5 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.490396 6648 ts_tablet_manager.cc:1916] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.491597 6153 tablet_service.cc:1511] Processing CreateTablet for tablet a30189879fbf49c796eea2fe621b9d9a (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.492020 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a30189879fbf49c796eea2fe621b9d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.492458 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 0457d931c271465e8d3c17461af639a5 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.492750 6649 tablet_replica.cc:333] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.492972 6649 raft_consensus.cc:2243] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.493228 6649 pending_rounds.cc:70] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Trying to abort 1 pending ops.
I20260806 07:58:28.493287 6649 pending_rounds.cc:77] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468730822033408 op_type: NO_OP noop_request { }
I20260806 07:58:28.493408 6649 raft_consensus.cc:2889] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:28.493518 6649 raft_consensus.cc:2272] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.497984 6649 ts_tablet_manager.cc:1916] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.498725 6537 tablet_bootstrap.cc:492] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.500483 6537 tablet_bootstrap.cc:654] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.502499 6537 tablet_bootstrap.cc:492] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.502671 6537 ts_tablet_manager.cc:1403] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 07:58:28.503436 6537 raft_consensus.cc:359] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.503631 6755 ts_tablet_manager.cc:1929] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.503652 6537 raft_consensus.cc:385] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.503731 6537 raft_consensus.cc:740] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.503893 6755 log.cc:1199] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/c0e540b33899435da8546e5080bfa40b
I20260806 07:58:28.504004 6537 consensus_queue.cc:260] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.504060 6294 tablet_service.cc:1511] Processing CreateTablet for tablet a30189879fbf49c796eea2fe621b9d9a (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.504395 6755 ts_tablet_manager.cc:1950] T c0e540b33899435da8546e5080bfa40b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.504562 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a30189879fbf49c796eea2fe621b9d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.505170 6537 ts_tablet_manager.cc:1434] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:28.505430 6755 tablet_replica.cc:333] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.505704 6755 raft_consensus.cc:2243] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.505903 6755 raft_consensus.cc:2272] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.506708 6755 ts_tablet_manager.cc:1916] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.510388 6649 ts_tablet_manager.cc:1929] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.510619 6649 log.cc:1199] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/46f5138fc13a493bb7217f123071c20a
I20260806 07:58:28.511193 6649 ts_tablet_manager.cc:1950] T 46f5138fc13a493bb7217f123071c20a P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.511301 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet c0e540b33899435da8546e5080bfa40b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.512001 6581 consensus_queue.cc:1048] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.512156 6648 ts_tablet_manager.cc:1929] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.512308 6648 log.cc:1199] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/4e093658a8714105a67c0165bc2978b1
I20260806 07:58:28.512336 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 46f5138fc13a493bb7217f123071c20a (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.512728 6649 tablet_replica.cc:333] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.512812 6648 ts_tablet_manager.cc:1950] T 4e093658a8714105a67c0165bc2978b1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.512946 6649 raft_consensus.cc:2243] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.513171 6649 raft_consensus.cc:2272] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.514258 6649 ts_tablet_manager.cc:1916] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.516592 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 4e093658a8714105a67c0165bc2978b1 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.517728 6532 tablet_bootstrap.cc:492] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.519654 6532 tablet_bootstrap.cc:654] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.519927 6755 ts_tablet_manager.cc:1929] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.520279 6755 log.cc:1199] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/8cea29d556ed4c2cb5fc9a8e4197bc7b
I20260806 07:58:28.520865 6755 ts_tablet_manager.cc:1950] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.521894 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.522007 6755 tablet_replica.cc:333] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.522256 6755 raft_consensus.cc:2243] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.522437 6755 raft_consensus.cc:2272] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.523504 6423 tablet_service.cc:1511] Processing CreateTablet for tablet a30189879fbf49c796eea2fe621b9d9a (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.524034 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a30189879fbf49c796eea2fe621b9d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.529438 6583 consensus_queue.cc:1048] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.530043 6755 ts_tablet_manager.cc:1916] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.532109 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.004s sys 0.000s Metrics: {"bytes_written":640,"delete_count":0,"lbm_write_time_us":153,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":13}
I20260806 07:58:28.539083 6428 tablet_service.cc:1511] Processing CreateTablet for tablet 054c7be39877456fb2b196b5a75610bb (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.539067 6147 tablet_service.cc:1511] Processing CreateTablet for tablet 3036ed4cf6f04ec88c613f1bec3255d0 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.539515 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054c7be39877456fb2b196b5a75610bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.539525 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3036ed4cf6f04ec88c613f1bec3255d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.543638 6531 tablet_bootstrap.cc:492] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.544090 6649 ts_tablet_manager.cc:1929] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.544250 6649 log.cc:1199] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/8cea29d556ed4c2cb5fc9a8e4197bc7b
I20260806 07:58:28.544821 6649 ts_tablet_manager.cc:1950] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.545872 6531 tablet_bootstrap.cc:654] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.546141 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.546526 6649 tablet_replica.cc:333] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.546566 6537 tablet_bootstrap.cc:492] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.546761 6649 raft_consensus.cc:2243] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.546999 6649 raft_consensus.cc:2272] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.547652 6153 tablet_service.cc:1511] Processing CreateTablet for tablet 054c7be39877456fb2b196b5a75610bb (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.548162 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054c7be39877456fb2b196b5a75610bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.548245 6649 ts_tablet_manager.cc:1916] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.548620 6537 tablet_bootstrap.cc:654] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.550019 6294 tablet_service.cc:1511] Processing CreateTablet for tablet 054c7be39877456fb2b196b5a75610bb (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.550467 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054c7be39877456fb2b196b5a75610bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.554656 6531 tablet_bootstrap.cc:492] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.554834 6531 ts_tablet_manager.cc:1403] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.004s
I20260806 07:58:28.555610 6531 raft_consensus.cc:359] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.557417 6531 raft_consensus.cc:385] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.557520 6531 raft_consensus.cc:740] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.557916 6531 consensus_queue.cc:260] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.558601 6531 ts_tablet_manager.cc:1434] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 07:58:28.558903 6531 tablet_bootstrap.cc:492] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.560046 6288 tablet_service.cc:1511] Processing CreateTablet for tablet 3036ed4cf6f04ec88c613f1bec3255d0 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.560489 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3036ed4cf6f04ec88c613f1bec3255d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.560590 6531 tablet_bootstrap.cc:654] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.562134 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.139s user 0.004s sys 0.010s Metrics: {"bytes_written":6939,"cfile_init":1,"dirs.queue_time_us":44,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":716,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":937,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260806 07:58:28.562963 6755 ts_tablet_manager.cc:1929] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.563120 6755 log.cc:1199] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/88ff8d074e9e4a7d9f788fb501528bae
I20260806 07:58:28.563223 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.563278 6649 ts_tablet_manager.cc:1929] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.563414 6649 log.cc:1199] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/c0e540b33899435da8546e5080bfa40b
I20260806 07:58:28.563578 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:28.563884 6649 ts_tablet_manager.cc:1950] T c0e540b33899435da8546e5080bfa40b P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.563908 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:28.564211 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.565025 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet c0e540b33899435da8546e5080bfa40b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.565562 6755 ts_tablet_manager.cc:1950] T 88ff8d074e9e4a7d9f788fb501528bae P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.566185 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4}
I20260806 07:58:28.566570 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.566787 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 88ff8d074e9e4a7d9f788fb501528bae (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.566851 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.567612 6649 tablet_replica.cc:333] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.567816 6649 raft_consensus.cc:2243] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.568020 6649 raft_consensus.cc:2272] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.569715 6649 ts_tablet_manager.cc:1916] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.569767 6755 tablet_replica.cc:333] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.569968 6755 raft_consensus.cc:2243] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.570163 6755 raft_consensus.cc:2272] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.570737 6755 ts_tablet_manager.cc:1916] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.571662 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.003s sys 0.002s Metrics: {"bytes_written":688,"delete_count":0,"lbm_write_time_us":192,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260806 07:58:28.572510 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.345208
I20260806 07:58:28.572849 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.345208
I20260806 07:58:28.577966 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.279703
I20260806 07:58:28.578701 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.584244 6649 ts_tablet_manager.cc:1929] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.584384 6649 log.cc:1199] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/f89b495f1733475895ce958ae57861e1
I20260806 07:58:28.584844 6649 ts_tablet_manager.cc:1950] T f89b495f1733475895ce958ae57861e1 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.589206 6648 tablet_replica.cc:333] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.589938 6648 raft_consensus.cc:2243] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.592312 6648 raft_consensus.cc:2272] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.593549 6532 tablet_bootstrap.cc:492] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.593851 6532 ts_tablet_manager.cc:1403] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.076s user 0.004s sys 0.006s
I20260806 07:58:28.594816 6532 raft_consensus.cc:359] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.595141 6532 raft_consensus.cc:385] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.595279 6532 raft_consensus.cc:740] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.595376 6648 ts_tablet_manager.cc:1916] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.595741 6532 consensus_queue.cc:260] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.596617 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1668,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":692,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":671,"lbm_read_time_us":368,"lbm_reads_lt_1ms":21,"lbm_write_time_us":4581,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"peak_mem_usage":32768,"reinsert_count":0,"update_count":16}
I20260806 07:58:28.597175 6755 ts_tablet_manager.cc:1929] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:28.597321 6755 log.cc:1199] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/46f5138fc13a493bb7217f123071c20a
I20260806 07:58:28.597723 6532 ts_tablet_manager.cc:1434] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:28.597817 6755 ts_tablet_manager.cc:1950] T 46f5138fc13a493bb7217f123071c20a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.598156 6532 tablet_bootstrap.cc:492] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.599009 6755 tablet_replica.cc:333] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.599231 6755 raft_consensus.cc:2243] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.599574 6755 raft_consensus.cc:2272] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.599704 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 46f5138fc13a493bb7217f123071c20a (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.600821 6428 tablet_service.cc:1511] Processing CreateTablet for tablet 3036ed4cf6f04ec88c613f1bec3255d0 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.601276 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3036ed4cf6f04ec88c613f1bec3255d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.601867 6423 tablet_service.cc:1511] Processing CreateTablet for tablet d2e9449a49ea445b89274f38ddca53ac (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.601898 6294 tablet_service.cc:1511] Processing CreateTablet for tablet d2e9449a49ea445b89274f38ddca53ac (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.601934 6147 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4bee6c27bf4e9198f13f81ea27678e (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.602263 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2e9449a49ea445b89274f38ddca53ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.602275 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2e9449a49ea445b89274f38ddca53ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.602352 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4bee6c27bf4e9198f13f81ea27678e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.607280 6153 tablet_service.cc:1511] Processing CreateTablet for tablet d2e9449a49ea445b89274f38ddca53ac (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.607784 6532 tablet_bootstrap.cc:654] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.608835 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2e9449a49ea445b89274f38ddca53ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.609390 6755 ts_tablet_manager.cc:1916] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.615856 6537 tablet_bootstrap.cc:492] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.616155 6537 ts_tablet_manager.cc:1403] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.070s user 0.002s sys 0.010s
I20260806 07:58:28.616446 6531 tablet_bootstrap.cc:492] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.616919 6531 ts_tablet_manager.cc:1403] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.010s
I20260806 07:58:28.618006 6531 raft_consensus.cc:359] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.618213 6531 raft_consensus.cc:385] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.618346 6531 raft_consensus.cc:740] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.618682 6531 consensus_queue.cc:260] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.619652 6288 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4bee6c27bf4e9198f13f81ea27678e (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.620117 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4bee6c27bf4e9198f13f81ea27678e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.625420 6285 tablet_service.cc:1511] Processing CreateTablet for tablet e4de34f839a442818e8c8045000f7cca (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.626005 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4de34f839a442818e8c8045000f7cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.626410 6428 tablet_service.cc:1511] Processing CreateTablet for tablet e4de34f839a442818e8c8045000f7cca (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.626814 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4de34f839a442818e8c8045000f7cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.628846 6755 ts_tablet_manager.cc:1929] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.628988 6755 log.cc:1199] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/0457d931c271465e8d3c17461af639a5
I20260806 07:58:28.629442 6755 ts_tablet_manager.cc:1950] T 0457d931c271465e8d3c17461af639a5 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.629953 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4bee6c27bf4e9198f13f81ea27678e (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.630349 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4bee6c27bf4e9198f13f81ea27678e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.631142 6755 tablet_replica.cc:333] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.631331 6755 raft_consensus.cc:2243] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.631361 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 0457d931c271465e8d3c17461af639a5 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.631529 6755 raft_consensus.cc:2272] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.632531 6755 ts_tablet_manager.cc:1916] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.636108 6531 ts_tablet_manager.cc:1434] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20260806 07:58:28.636435 6531 tablet_bootstrap.cc:492] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
W20260806 07:58:28.637102 6384 consensus_peers.cc:597] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.638065 6285 tablet_service.cc:1511] Processing CreateTablet for tablet d50d72195b4743d194125a7b231fcc80 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.638366 6531 tablet_bootstrap.cc:654] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.638907 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d72195b4743d194125a7b231fcc80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.640537 6537 raft_consensus.cc:359] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.640756 6537 raft_consensus.cc:385] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.640946 6160 tablet_service.cc:1511] Processing CreateTablet for tablet d50d72195b4743d194125a7b231fcc80 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.641434 6649 tablet_replica.cc:333] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.641566 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet f89b495f1733475895ce958ae57861e1 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.641690 6649 raft_consensus.cc:2243] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.642035 6649 raft_consensus.cc:2272] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.642678 6755 ts_tablet_manager.cc:1929] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.642810 6755 log.cc:1199] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/4e093658a8714105a67c0165bc2978b1
I20260806 07:58:28.643075 6532 tablet_bootstrap.cc:492] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.643112 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d72195b4743d194125a7b231fcc80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.643234 6532 ts_tablet_manager.cc:1403] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.002s
I20260806 07:58:28.643317 6755 ts_tablet_manager.cc:1950] T 4e093658a8714105a67c0165bc2978b1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.644048 6532 raft_consensus.cc:359] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.644155 6537 raft_consensus.cc:740] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.644258 6532 raft_consensus.cc:385] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.644356 6532 raft_consensus.cc:740] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.644637 6532 consensus_queue.cc:260] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.644977 6537 consensus_queue.cc:260] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.646037 6537 ts_tablet_manager.cc:1434] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.030s user 0.003s sys 0.000s
I20260806 07:58:28.646346 6532 ts_tablet_manager.cc:1434] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:28.646402 6537 tablet_bootstrap.cc:492] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.646669 6532 tablet_bootstrap.cc:492] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.648891 6532 tablet_bootstrap.cc:654] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.649573 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 4e093658a8714105a67c0165bc2978b1 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.650020 6755 tablet_replica.cc:333] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.650223 6755 raft_consensus.cc:2243] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.650381 6755 raft_consensus.cc:2272] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.652266 6755 ts_tablet_manager.cc:1916] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.644038 6147 tablet_service.cc:1511] Processing CreateTablet for tablet e4de34f839a442818e8c8045000f7cca (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.654165 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4de34f839a442818e8c8045000f7cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.658006 6532 tablet_bootstrap.cc:492] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.658181 6532 ts_tablet_manager.cc:1403] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260806 07:58:28.658573 6531 tablet_bootstrap.cc:492] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.658764 6531 ts_tablet_manager.cc:1403] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260806 07:58:28.658977 6532 raft_consensus.cc:359] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.659192 6532 raft_consensus.cc:385] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.659277 6532 raft_consensus.cc:740] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.659579 6532 consensus_queue.cc:260] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.659621 6531 raft_consensus.cc:359] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.659825 6531 raft_consensus.cc:385] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.659914 6531 raft_consensus.cc:740] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.660558 6531 consensus_queue.cc:260] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.662489 6531 ts_tablet_manager.cc:1434] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:28.662838 6531 tablet_bootstrap.cc:492] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.664776 6532 ts_tablet_manager.cc:1434] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 07:58:28.665249 6531 tablet_bootstrap.cc:654] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.667151 5931 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:28.669169 6648 ts_tablet_manager.cc:1929] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.669411 6648 log.cc:1199] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/88ff8d074e9e4a7d9f788fb501528bae
I20260806 07:58:28.669941 6648 ts_tablet_manager.cc:1950] T 88ff8d074e9e4a7d9f788fb501528bae P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.671002 6532 tablet_bootstrap.cc:492] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:28.672180 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.093s user 0.014s sys 0.023s Metrics: {"dirs.queue_time_us":2658,"dirs.run_cpu_time_us":964,"dirs.run_wall_time_us":32507,"lbm_write_time_us":362,"lbm_writes_lt_1ms":7,"mutex_wait_us":415,"num_input_rowsets":0,"thread_start_us":70266,"threads_started":2}
I20260806 07:58:28.672782 6532 tablet_bootstrap.cc:654] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.673059 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.365049
I20260806 07:58:28.673208 6538 consensus_queue.cc:1048] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.675189 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.365049
I20260806 07:58:28.675750 6531 tablet_bootstrap.cc:492] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.675915 6531 ts_tablet_manager.cc:1403] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260806 07:58:28.676036 6423 tablet_service.cc:1511] Processing CreateTablet for tablet d50d72195b4743d194125a7b231fcc80 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.676487 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d72195b4743d194125a7b231fcc80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.676761 6531 raft_consensus.cc:359] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.676952 6531 raft_consensus.cc:385] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.677048 6531 raft_consensus.cc:740] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.677289 6531 consensus_queue.cc:260] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.677887 6531 ts_tablet_manager.cc:1434] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:28.678218 6531 tablet_bootstrap.cc:492] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.679979 6531 tablet_bootstrap.cc:654] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.684096 6755 ts_tablet_manager.cc:1929] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.684250 6755 log.cc:1199] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/e1c0ddbb77d94c408ed054fd4dbdcf4e
I20260806 07:58:28.684661 6755 ts_tablet_manager.cc:1950] T e1c0ddbb77d94c408ed054fd4dbdcf4e P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:28.686161 6649 ts_tablet_manager.cc:1916] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.688920 6428 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.689465 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.697080 5931 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:28.696972 6294 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.697561 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.673709 6583 raft_consensus.cc:493] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.703976 6583 raft_consensus.cc:515] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:28.704469 6387 consensus_peers.cc:597] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 -> Peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Couldn't send request to peer 7793159fc1ab4bd1b2a50dbd3b3e5168. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e093658a8714105a67c0165bc2978b1. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.705293 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a30189879fbf49c796eea2fe621b9d9a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.705485 6450 raft_consensus.cc:2468] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.706712 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a30189879fbf49c796eea2fe621b9d9a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.706991 6317 raft_consensus.cc:2468] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.708113 6422 tablet_service.cc:1511] Processing CreateTablet for tablet de9192eb384944ba9836187ba27edb38 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.708586 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de9192eb384944ba9836187ba27edb38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.709349 6288 tablet_service.cc:1511] Processing CreateTablet for tablet 33b87b2e51024e3d8311bbac25ae573b (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.709795 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b87b2e51024e3d8311bbac25ae573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.714002 6430 tablet_service.cc:1511] Processing CreateTablet for tablet 33b87b2e51024e3d8311bbac25ae573b (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.714310 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet e1c0ddbb77d94c408ed054fd4dbdcf4e (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.714388 6430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b87b2e51024e3d8311bbac25ae573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.714932 6755 tablet_replica.cc:333] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:28.715163 6755 raft_consensus.cc:2243] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.715518 6755 raft_consensus.cc:2272] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.718293 6288 tablet_service.cc:1511] Processing CreateTablet for tablet f054c628591a4ef7ab8d607abd77a984 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.720213 6425 tablet_service.cc:1511] Processing CreateTablet for tablet f054c628591a4ef7ab8d607abd77a984 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.720697 6425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f054c628591a4ef7ab8d607abd77a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.725409 6649 ts_tablet_manager.cc:1929] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.725556 6649 log.cc:1199] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/59cd956622e64e7fbfd27f370d63605b
I20260806 07:58:28.726071 6649 ts_tablet_manager.cc:1950] T 59cd956622e64e7fbfd27f370d63605b P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.726770 6422 tablet_service.cc:1511] Processing CreateTablet for tablet b5fe3aedfb004f9185b6307e0c087b63 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.727237 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fe3aedfb004f9185b6307e0c087b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.728183 6430 tablet_service.cc:1511] Processing CreateTablet for tablet 39fcd48f42fc4b0a9edc04adcc8364a0 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.730872 6427 tablet_service.cc:1511] Processing CreateTablet for tablet fd1e9ae4d73d46d99f50b701d63d4f18 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.731418 6427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1e9ae4d73d46d99f50b701d63d4f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.732451 6649 tablet_replica.cc:333] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.732658 6649 raft_consensus.cc:2243] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.732875 6649 raft_consensus.cc:2272] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.732898 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 59cd956622e64e7fbfd27f370d63605b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.734427 6421 tablet_service.cc:1511] Processing CreateTablet for tablet 43e15cc19276474a8b160d7959d8bf89 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.734838 6421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e15cc19276474a8b160d7959d8bf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.739784 6428 tablet_service.cc:1511] Processing CreateTablet for tablet 7d68050d52894d868b86370cd5e449b8 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.740371 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d68050d52894d868b86370cd5e449b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.742322 6430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39fcd48f42fc4b0a9edc04adcc8364a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.742455 6153 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.744626 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f054c628591a4ef7ab8d607abd77a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.745749 6755 ts_tablet_manager.cc:1916] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.746155 6149 tablet_service.cc:1511] Processing CreateTablet for tablet 39fcd48f42fc4b0a9edc04adcc8364a0 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.746980 6148 tablet_service.cc:1511] Processing CreateTablet for tablet 43e15cc19276474a8b160d7959d8bf89 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.747349 6296 tablet_service.cc:1511] Processing CreateTablet for tablet 7d68050d52894d868b86370cd5e449b8 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.747536 6156 tablet_service.cc:1511] Processing CreateTablet for tablet 7d68050d52894d868b86370cd5e449b8 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.749581 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.068s user 0.003s sys 0.004s Metrics: {"dirs.queue_time_us":9470,"dirs.run_cpu_time_us":841,"dirs.run_wall_time_us":5580,"lbm_write_time_us":324,"lbm_writes_lt_1ms":7,"mutex_wait_us":8973,"num_input_rowsets":0}
I20260806 07:58:28.749840 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 88ff8d074e9e4a7d9f788fb501528bae (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.752211 6649 ts_tablet_manager.cc:1916] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.752914 6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d68050d52894d868b86370cd5e449b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.753293 6583 leader_election.cc:290] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.753721 6295 tablet_service.cc:1511] Processing CreateTablet for tablet b5fe3aedfb004f9185b6307e0c087b63 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.754186 6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fe3aedfb004f9185b6307e0c087b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.753527 6114 leader_election.cc:304] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
W20260806 07:58:28.754846 6114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.202.195:36665, user_credentials={real_user=slave}} blocked reactor thread for 48835.7us
I20260806 07:58:28.755736 6544 raft_consensus.cc:2804] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.755847 6544 raft_consensus.cc:493] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.755927 6544 raft_consensus.cc:3060] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.758867 6544 raft_consensus.cc:515] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.760751 6583 raft_consensus.cc:493] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.760953 6583 raft_consensus.cc:515] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.761562 6583 leader_election.cc:290] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.762476 6531 tablet_bootstrap.cc:492] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.762643 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a30189879fbf49c796eea2fe621b9d9a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.762897 6450 raft_consensus.cc:3060] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.763492 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260806 07:58:28.763624 6113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.202.194:32979, user_credentials={real_user=slave}} blocked reactor thread for 54434.8us
I20260806 07:58:28.765792 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39fcd48f42fc4b0a9edc04adcc8364a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.766697 6160 tablet_service.cc:1511] Processing CreateTablet for tablet de9192eb384944ba9836187ba27edb38 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.767608 6147 tablet_service.cc:1511] Processing CreateTablet for tablet 33b87b2e51024e3d8311bbac25ae573b (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.768002 6155 tablet_service.cc:1511] Processing CreateTablet for tablet b5fe3aedfb004f9185b6307e0c087b63 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.768356 6146 tablet_service.cc:1511] Processing CreateTablet for tablet fd1e9ae4d73d46d99f50b701d63d4f18 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.768631 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2636 bytes on disk
I20260806 07:58:28.768608 6152 tablet_service.cc:1511] Processing CreateTablet for tablet f054c628591a4ef7ab8d607abd77a984 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.768903 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2636 bytes on disk
I20260806 07:58:28.769104 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2636 bytes on disk
I20260806 07:58:28.769744 6156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d68050d52894d868b86370cd5e449b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.769760 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":90,"lbm_reads_lt_1ms":4}
I20260806 07:58:28.770206 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.770470 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.773352 6544 leader_election.cc:290] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.773559 6544 consensus_queue.cc:1048] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.773763 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e15cc19276474a8b160d7959d8bf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.774350 6648 tablet_replica.cc:333] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.774610 6648 raft_consensus.cc:2243] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.774837 6648 raft_consensus.cc:2272] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.775416 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de9192eb384944ba9836187ba27edb38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.776137 6648 ts_tablet_manager.cc:1916] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.778245 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b87b2e51024e3d8311bbac25ae573b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.779937 6450 raft_consensus.cc:2468] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.780032 6537 tablet_bootstrap.cc:654] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.781298 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3036ed4cf6f04ec88c613f1bec3255d0" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.781567 6451 raft_consensus.cc:2468] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.782021 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fe3aedfb004f9185b6307e0c087b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.782773 6622 raft_consensus.cc:493] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.782951 6622 raft_consensus.cc:515] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.783630 6622 leader_election.cc:290] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.785878 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1e9ae4d73d46d99f50b701d63d4f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.787791 6648 ts_tablet_manager.cc:1929] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.787930 6648 log.cc:1199] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/0457d931c271465e8d3c17461af639a5
I20260806 07:58:28.788488 6648 ts_tablet_manager.cc:1950] T 0457d931c271465e8d3c17461af639a5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.789680 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 0457d931c271465e8d3c17461af639a5 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.789819 6152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f054c628591a4ef7ab8d607abd77a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.792888 6537 tablet_bootstrap.cc:492] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.793063 6537 ts_tablet_manager.cc:1403] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.147s user 0.002s sys 0.002s
I20260806 07:58:28.793919 6537 raft_consensus.cc:359] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.762665 6531 ts_tablet_manager.cc:1403] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.084s user 0.023s sys 0.046s
I20260806 07:58:28.794131 6537 raft_consensus.cc:385] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.794211 6537 raft_consensus.cc:740] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.794442 6537 consensus_queue.cc:260] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.794956 6531 raft_consensus.cc:359] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.795105 6537 ts_tablet_manager.cc:1434] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:28.795182 6531 raft_consensus.cc:385] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.795271 6531 raft_consensus.cc:740] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.795434 6537 tablet_bootstrap.cc:492] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.795557 6531 consensus_queue.cc:260] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.797230 6537 tablet_bootstrap.cc:654] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.797662 6531 ts_tablet_manager.cc:1434] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:28.797966 6531 tablet_bootstrap.cc:492] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.798285 6648 tablet_replica.cc:333] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.798550 6648 raft_consensus.cc:2243] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.798730 6648 raft_consensus.cc:2272] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.799351 6648 ts_tablet_manager.cc:1916] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.800074 6531 tablet_bootstrap.cc:654] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.801517 6114 leader_election.cc:304] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.801877 6544 raft_consensus.cc:2804] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.801909 6114 leader_election.cc:304] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.802067 6544 raft_consensus.cc:697] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:28.802335 6581 raft_consensus.cc:2804] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.802443 6581 raft_consensus.cc:493] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.802546 6581 raft_consensus.cc:3060] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.802358 6544 consensus_queue.cc:237] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.805253 6581 raft_consensus.cc:515] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.811762 6583 consensus_queue.cc:1048] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.819701 6537 tablet_bootstrap.cc:492] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.819877 6537 ts_tablet_manager.cc:1403] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20260806 07:58:28.820317 6581 leader_election.cc:290] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.821720 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3036ed4cf6f04ec88c613f1bec3255d0" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:28.821987 6453 raft_consensus.cc:3060] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.822450 6544 consensus_queue.cc:1048] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.823195 6537 raft_consensus.cc:359] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.823383 6537 raft_consensus.cc:385] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.823490 6537 raft_consensus.cc:740] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.823724 6537 consensus_queue.cc:260] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.826027 6453 raft_consensus.cc:2468] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:28.828578 6648 ts_tablet_manager.cc:1929] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:28.828717 6648 log.cc:1199] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/46f5138fc13a493bb7217f123071c20a
I20260806 07:58:28.829123 6648 ts_tablet_manager.cc:1950] T 46f5138fc13a493bb7217f123071c20a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.830482 6114 leader_election.cc:304] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.831755 6581 raft_consensus.cc:2804] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.831612 6016 catalog_manager.cc:5697] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.831918 6581 raft_consensus.cc:697] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:28.832152 6581 consensus_queue.cc:237] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.833736 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054c7be39877456fb2b196b5a75610bb" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.834028 6179 raft_consensus.cc:2468] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:28.835458 6387 leader_election.cc:304] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.834033 6537 ts_tablet_manager.cc:1434] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260806 07:58:28.836097 6537 tablet_bootstrap.cc:492] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.836500 6622 raft_consensus.cc:2804] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.836628 6622 raft_consensus.cc:493] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.836747 6622 raft_consensus.cc:3060] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.838807 6531 tablet_bootstrap.cc:492] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:28.838963 6531 ts_tablet_manager.cc:1403] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.009s
I20260806 07:58:28.839320 6537 tablet_bootstrap.cc:654] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.839758 6531 raft_consensus.cc:359] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.839920 6531 raft_consensus.cc:385] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.840705 6293 tablet_service.cc:1511] Processing CreateTablet for tablet 39fcd48f42fc4b0a9edc04adcc8364a0 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.841344 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39fcd48f42fc4b0a9edc04adcc8364a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.840004 6531 raft_consensus.cc:740] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.842166 6531 consensus_queue.cc:260] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.842720 6531 ts_tablet_manager.cc:1434] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260806 07:58:28.843016 6531 tablet_bootstrap.cc:492] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:28.843568 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2873 bytes on disk
I20260806 07:58:28.843907 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2873 bytes on disk
I20260806 07:58:28.844182 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 2873 bytes on disk
I20260806 07:58:28.844341 6284 tablet_service.cc:1511] Processing CreateTablet for tablet 43e15cc19276474a8b160d7959d8bf89 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.844780 6284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e15cc19276474a8b160d7959d8bf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.845008 6531 tablet_bootstrap.cc:654] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.844664 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.402s user 0.035s sys 0.013s Metrics: {"bytes_written":688,"cfile_cache_hit":15,"cfile_cache_hit_bytes":15304,"cfile_cache_miss":41,"cfile_cache_miss_bytes":24188,"cfile_init":12,"delete_count":0,"delta_iterators_relevant":12,"dirs.queue_time_us":10720,"dirs.run_cpu_time_us":1046,"dirs.run_wall_time_us":25632,"drs_written":1,"lbm_read_time_us":12958,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":88,"lbm_write_time_us":1196,"lbm_writes_lt_1ms":31,"mutex_wait_us":241,"num_input_rowsets":2,"peak_mem_usage":65536,"reinsert_count":0,"rows_written":54,"thread_start_us":77768,"threads_started":2,"update_count":16}
I20260806 07:58:28.845933 6287 tablet_service.cc:1511] Processing CreateTablet for tablet fd1e9ae4d73d46d99f50b701d63d4f18 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.846416 6287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1e9ae4d73d46d99f50b701d63d4f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.848661 6622 raft_consensus.cc:515] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.849848 6537 tablet_bootstrap.cc:492] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.850011 6537 ts_tablet_manager.cc:1403] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260806 07:58:28.850059 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260806 07:58:28.850466 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.850669 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.850800 6537 raft_consensus.cc:359] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.850997 6537 raft_consensus.cc:385] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.851080 6537 raft_consensus.cc:740] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.851347 6537 consensus_queue.cc:260] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.851869 6537 ts_tablet_manager.cc:1434] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:28.852195 6537 tablet_bootstrap.cc:492] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.854676 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 46f5138fc13a493bb7217f123071c20a (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.854974 6537 tablet_bootstrap.cc:654] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.855826 6648 tablet_replica.cc:333] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.856026 6648 raft_consensus.cc:2243] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.856389 6648 raft_consensus.cc:2272] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.856472 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.856755 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.857075 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.857335 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.858300 6648 ts_tablet_manager.cc:1916] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.859486 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.860457 6622 leader_election.cc:290] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.862738 6016 catalog_manager.cc:5697] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:28.863282 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":3585}
I20260806 07:58:28.864014 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.864306 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.866185 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.866513 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.866796 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.867389 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.868688 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.868969 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.869738 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.870764 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":152,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.872922 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":927,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":127,"lbm_reads_lt_1ms":8,"lbm_write_time_us":151,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:28.881119 6537 tablet_bootstrap.cc:492] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.881292 6537 ts_tablet_manager.cc:1403] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.003s
I20260806 07:58:28.882139 6537 raft_consensus.cc:359] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.882335 6537 raft_consensus.cc:385] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.882427 6537 raft_consensus.cc:740] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:28.882709 6537 consensus_queue.cc:260] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.883318 6537 ts_tablet_manager.cc:1434] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:28.883711 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.884017 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.884250 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.884434 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.884632 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054c7be39877456fb2b196b5a75610bb" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:28.884876 6179 raft_consensus.cc:3060] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.886015 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.886287 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.886507 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.886982 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.887380 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:28.860965 6649 ts_tablet_manager.cc:1929] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.890027 6649 log.cc:1199] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/aacb42bf87304fbe8aa1edf4cc6a9e58
I20260806 07:58:28.890689 6649 ts_tablet_manager.cc:1950] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.891601 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.891892 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.892074 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet aacb42bf87304fbe8aa1edf4cc6a9e58 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.892172 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.892453 6649 tablet_replica.cc:333] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.892452 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.892691 6649 raft_consensus.cc:2243] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.892707 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.892890 6649 raft_consensus.cc:2272] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.894268 6649 ts_tablet_manager.cc:1916] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.895797 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":8023}
I20260806 07:58:28.896082 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":202,"lbm_writes_lt_1ms":4,"mutex_wait_us":2476,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.896605 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.896899 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:28.900792 6649 ts_tablet_manager.cc:1929] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.900938 6649 log.cc:1199] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/8961173cf8d24b8eb022e994f90970da
I20260806 07:58:28.901381 6649 ts_tablet_manager.cc:1950] T 8961173cf8d24b8eb022e994f90970da P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.905030 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.005s sys 0.002s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":58,"lbm_read_time_us":300,"lbm_reads_lt_1ms":12,"lbm_write_time_us":208,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:28.905520 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.005s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":174,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.906026 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:28.906629 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 8961173cf8d24b8eb022e994f90970da (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.908823 6649 tablet_replica.cc:333] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.909353 6538 consensus_queue.cc:1048] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.145s
I20260806 07:58:28.909888 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":146,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:28.910372 6649 raft_consensus.cc:2243] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.907127 6285 tablet_service.cc:1511] Processing CreateTablet for tablet de9192eb384944ba9836187ba27edb38 (DEFAULT_TABLE table=table-2 [id=3fc3653402c94f3e85bec250c0e53983]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:28.911646 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.003s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1004,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":50,"lbm_read_time_us":153,"lbm_reads_lt_1ms":8,"lbm_write_time_us":163,"lbm_writes_lt_1ms":4,"mutex_wait_us":4282,"reinsert_count":0,"update_count":18}
I20260806 07:58:28.914688 6572 consensus_queue.cc:1048] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 07:58:28.914973 6649 raft_consensus.cc:2272] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.925200 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de9192eb384944ba9836187ba27edb38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:28.925923 6779 raft_consensus.cc:493] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.933041 6649 ts_tablet_manager.cc:1916] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.938953 6537 tablet_bootstrap.cc:492] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:28.939703 6581 consensus_queue.cc:1048] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:28.940136 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054c7be39877456fb2b196b5a75610bb" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.940608 6649 ts_tablet_manager.cc:1929] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.940723 6649 log.cc:1199] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/88ff8d074e9e4a7d9f788fb501528bae
I20260806 07:58:28.940857 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a30189879fbf49c796eea2fe621b9d9a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:28.941043 6317 raft_consensus.cc:3060] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.941160 6649 ts_tablet_manager.cc:1950] T 88ff8d074e9e4a7d9f788fb501528bae P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.941887 6572 consensus_queue.cc:1048] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260806 07:58:28.942286 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 88ff8d074e9e4a7d9f788fb501528bae (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.942652 6649 tablet_replica.cc:333] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.942860 6649 raft_consensus.cc:2243] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.943181 6649 raft_consensus.cc:2272] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.944061 6649 ts_tablet_manager.cc:1916] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.947151 6179 raft_consensus.cc:2468] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:28.947745 6583 consensus_queue.cc:1048] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260806 07:58:28.948853 6387 leader_election.cc:304] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.949215 6568 raft_consensus.cc:2804] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:28.949399 6568 raft_consensus.cc:697] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:28.949824 6568 consensus_queue.cc:237] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.952704 6649 ts_tablet_manager.cc:1929] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.952839 6649 log.cc:1199] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/4e093658a8714105a67c0165bc2978b1
I20260806 07:58:28.953245 6649 ts_tablet_manager.cc:1950] T 4e093658a8714105a67c0165bc2978b1 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:28.954296 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 4e093658a8714105a67c0165bc2978b1 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.955565 6649 tablet_replica.cc:333] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:28.955804 6649 raft_consensus.cc:2243] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:28.955967 6649 pending_rounds.cc:70] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Trying to abort 1 pending ops.
I20260806 07:58:28.956029 6649 pending_rounds.cc:77] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468731528818688 op_type: NO_OP noop_request { }
I20260806 07:58:28.956148 6649 raft_consensus.cc:2889] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:28.956247 6649 raft_consensus.cc:2272] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.957124 6649 ts_tablet_manager.cc:1916] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.926100 6779 raft_consensus.cc:515] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.961743 6532 tablet_bootstrap.cc:492] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:28.961784 6538 raft_consensus.cc:493] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.961397 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3036ed4cf6f04ec88c613f1bec3255d0" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:28.961956 6532 ts_tablet_manager.cc:1403] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.292s user 0.012s sys 0.046s
I20260806 07:58:28.961964 6538 raft_consensus.cc:515] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.966285 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4de34f839a442818e8c8045000f7cca" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:28.966513 6452 raft_consensus.cc:2468] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:28.967010 6114 leader_election.cc:304] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.967346 6581 raft_consensus.cc:2804] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.967952 6779 leader_election.cc:290] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.968175 6648 ts_tablet_manager.cc:1929] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.968289 6648 log.cc:1199] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/8cea29d556ed4c2cb5fc9a8e4197bc7b
I20260806 07:58:28.968662 6648 ts_tablet_manager.cc:1950] T 8cea29d556ed4c2cb5fc9a8e4197bc7b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:28.969844 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 8cea29d556ed4c2cb5fc9a8e4197bc7b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:28.970077 6581 raft_consensus.cc:493] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.970184 6581 raft_consensus.cc:3060] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.973091 6581 raft_consensus.cc:515] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.973660 6537 tablet_bootstrap.cc:654] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:28.977321 6316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3036ed4cf6f04ec88c613f1bec3255d0" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:28.978223 6622 raft_consensus.cc:493] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:28.978458 6648 tablet_replica.cc:333] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:28.978434 6622 raft_consensus.cc:515] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.978693 6648 raft_consensus.cc:2243] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:28.979051 6648 raft_consensus.cc:2272] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:28.980109 6622 leader_election.cc:290] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:28.980464 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4bee6c27bf4e9198f13f81ea27678e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:28.980660 6177 raft_consensus.cc:2468] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:28.981118 6387 leader_election.cc:304] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:28.981671 6622 raft_consensus.cc:2804] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:28.981776 6622 raft_consensus.cc:493] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:28.981895 6622 raft_consensus.cc:3060] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.982704 6318 raft_consensus.cc:2468] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:28.985101 6648 ts_tablet_manager.cc:1916] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:28.987607 6581 leader_election.cc:290] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:28.988238 6622 raft_consensus.cc:515] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:28.989566 6314 raft_consensus.cc:3060] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.989792 6532 raft_consensus.cc:359] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:28.994820 6250 consensus_peers.cc:597] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:28.996622 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4bee6c27bf4e9198f13f81ea27678e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:28.996853 6177 raft_consensus.cc:3060] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:28.997785 6532 raft_consensus.cc:385] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:28.992136 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4bee6c27bf4e9198f13f81ea27678e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:28.998533 6384 leader_election.cc:343] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:28.998904 6537 tablet_bootstrap.cc:492] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:28.999096 6537 ts_tablet_manager.cc:1403] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.082s user 0.003s sys 0.001s
I20260806 07:58:28.999941 6537 raft_consensus.cc:359] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.000154 6537 raft_consensus.cc:385] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.000226 6537 raft_consensus.cc:740] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.000295 6532 raft_consensus.cc:740] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.000538 6177 raft_consensus.cc:2468] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.000511 6537 consensus_queue.cc:260] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.000522 6532 consensus_queue.cc:260] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.000970 6387 leader_election.cc:304] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.001112 6537 ts_tablet_manager.cc:1434] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:29.001292 6568 raft_consensus.cc:2804] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.001425 6568 raft_consensus.cc:697] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:29.001442 6537 tablet_bootstrap.cc:492] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.002435 6532 ts_tablet_manager.cc:1434] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.039s user 0.001s sys 0.005s
I20260806 07:58:29.002766 6532 tablet_bootstrap.cc:492] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.003443 6537 tablet_bootstrap.cc:654] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.003767 6568 consensus_queue.cc:237] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.005923 6648 ts_tablet_manager.cc:1929] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.006070 6648 log.cc:1199] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/f89b495f1733475895ce958ae57861e1
I20260806 07:58:29.007663 6648 ts_tablet_manager.cc:1950] T f89b495f1733475895ce958ae57861e1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.009615 6648 tablet_replica.cc:333] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.009747 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet f89b495f1733475895ce958ae57861e1 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.009835 6648 raft_consensus.cc:2243] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.010040 6648 raft_consensus.cc:2272] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.003211 6313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054c7be39877456fb2b196b5a75610bb" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:29.012042 6313 raft_consensus.cc:3060] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.012848 6649 ts_tablet_manager.cc:1929] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:28.992671 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4de34f839a442818e8c8045000f7cca" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.013161 6451 raft_consensus.cc:3060] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.012992 6649 log.cc:1199] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/06cc4788571140b2ab78811966d2e06d
I20260806 07:58:29.015904 6649 ts_tablet_manager.cc:1950] T 06cc4788571140b2ab78811966d2e06d P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.017313 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 06cc4788571140b2ab78811966d2e06d (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.017580 6532 tablet_bootstrap.cc:654] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.018362 6622 leader_election.cc:290] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.019008 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4bee6c27bf4e9198f13f81ea27678e" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:29.019696 6384 leader_election.cc:343] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.020890 6649 tablet_replica.cc:333] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.023836 6451 raft_consensus.cc:2468] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.024453 6114 leader_election.cc:304] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.024870 6581 raft_consensus.cc:2804] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.025028 6581 raft_consensus.cc:697] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:29.025415 6581 consensus_queue.cc:237] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.036602 6317 raft_consensus.cc:2468] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.039614 6568 raft_consensus.cc:493] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.039829 6568 raft_consensus.cc:515] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.041487 6568 leader_election.cc:290] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.041980 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2e9449a49ea445b89274f38ddca53ac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.042196 6177 raft_consensus.cc:2468] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.042773 6387 leader_election.cc:304] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.043428 6568 raft_consensus.cc:2804] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.043543 6568 raft_consensus.cc:493] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.043655 6568 raft_consensus.cc:3060] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.045885 6649 raft_consensus.cc:2243] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.046602 6649 pending_rounds.cc:70] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Trying to abort 1 pending ops.
I20260806 07:58:29.046687 6649 pending_rounds.cc:77] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468731640086528 op_type: NO_OP noop_request { }
I20260806 07:58:29.046833 6649 raft_consensus.cc:2889] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:29.046963 6649 raft_consensus.cc:2272] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.047923 6649 ts_tablet_manager.cc:1916] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.048269 6531 tablet_bootstrap.cc:492] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.048496 6531 ts_tablet_manager.cc:1403] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.206s user 0.019s sys 0.041s
I20260806 07:58:29.049414 6531 raft_consensus.cc:359] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.049666 6531 raft_consensus.cc:385] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.049759 6531 raft_consensus.cc:740] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.050009 6016 catalog_manager.cc:5697] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.051394 6648 ts_tablet_manager.cc:1916] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.054994 6649 ts_tablet_manager.cc:1929] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.055174 6649 log.cc:1199] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/462655eb372b4284a2b88378bbb0f686
I20260806 07:58:29.055178 6014 catalog_manager.cc:5697] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.055619 6649 ts_tablet_manager.cc:1950] T 462655eb372b4284a2b88378bbb0f686 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.055905 6014 catalog_manager.cc:5697] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.056764 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 462655eb372b4284a2b88378bbb0f686 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.057009 6649 tablet_replica.cc:333] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.057197 6649 raft_consensus.cc:2243] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.057375 6649 raft_consensus.cc:2272] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.058113 6531 consensus_queue.cc:260] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.058313 6649 ts_tablet_manager.cc:1916] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.058580 6623 raft_consensus.cc:493] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.058750 6623 raft_consensus.cc:515] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.058996 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.386s user 0.027s sys 0.067s Metrics: {"bytes_written":25510,"cfile_cache_hit":16,"cfile_cache_hit_bytes":19132,"cfile_cache_miss":40,"cfile_cache_miss_bytes":27776,"cfile_init":11,"delta_iterators_relevant":12,"dirs.queue_time_us":4932,"dirs.run_cpu_time_us":1289,"dirs.run_wall_time_us":40877,"drs_written":1,"lbm_read_time_us":1696,"lbm_reads_lt_1ms":84,"lbm_write_time_us":1194,"lbm_writes_lt_1ms":27,"mutex_wait_us":282,"num_input_rowsets":2,"peak_mem_usage":65536,"rows_written":54,"spinlock_wait_cycles":768,"thread_start_us":224317,"threads_started":2}
I20260806 07:58:29.059482 6623 leader_election.cc:290] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.058755 6531 ts_tablet_manager.cc:1434] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260806 07:58:29.059909 6531 tablet_bootstrap.cc:492] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.060261 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3dc4525dae4b5f83c33b0be1ac01f6" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:29.061149 6387 leader_election.cc:343] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.061751 6531 tablet_bootstrap.cc:654] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.066520 6537 tablet_bootstrap.cc:492] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.066670 6537 ts_tablet_manager.cc:1403] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.065s user 0.017s sys 0.017s
I20260806 07:58:29.067457 6537 raft_consensus.cc:359] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.067626 6537 raft_consensus.cc:385] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.067710 6537 raft_consensus.cc:740] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.067900 6537 consensus_queue.cc:260] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.068737 6537 ts_tablet_manager.cc:1434] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:29.069015 6537 tablet_bootstrap.cc:492] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.070377 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.070634 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.070834 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.071065 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.071667 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.071950 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.072196 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.072669 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.073030 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.073144 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.073331 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.073323 6648 ts_tablet_manager.cc:1929] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.073496 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.073522 6648 log.cc:1199] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/59cd956622e64e7fbfd27f370d63605b
I20260806 07:58:29.073774 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.074062 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.074286 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.074509 6531 tablet_bootstrap.cc:492] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.074743 6531 ts_tablet_manager.cc:1403] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20260806 07:58:29.075917 6649 ts_tablet_manager.cc:1929] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.076036 6649 log.cc:1199] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/81dd62212ec041f19f9760efb21ded44
I20260806 07:58:29.076159 6531 raft_consensus.cc:359] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.076378 6531 raft_consensus.cc:385] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.076499 6649 ts_tablet_manager.cc:1950] T 81dd62212ec041f19f9760efb21ded44 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.076527 6531 raft_consensus.cc:740] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.076676 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.076815 6531 consensus_queue.cc:260] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.077371 6531 ts_tablet_manager.cc:1434] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:29.077665 6649 tablet_replica.cc:333] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.077721 6531 tablet_bootstrap.cc:492] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.077888 6649 raft_consensus.cc:2243] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.078070 6649 raft_consensus.cc:2272] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.078104 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.078377 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.078482 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.078723 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.079049 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.078992 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":177,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.079453 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.079787 6531 tablet_bootstrap.cc:654] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.084702 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 81dd62212ec041f19f9760efb21ded44 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.085911 6648 ts_tablet_manager.cc:1950] T 59cd956622e64e7fbfd27f370d63605b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.087157 6648 tablet_replica.cc:333] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.087379 6648 raft_consensus.cc:2243] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.087567 6648 raft_consensus.cc:2272] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.089001 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 59cd956622e64e7fbfd27f370d63605b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.089257 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.089380 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3036ed4cf6f04ec88c613f1bec3255d0" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.089675 6453 raft_consensus.cc:2393] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:29.089691 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.090003 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.090536 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":11858}
I20260806 07:58:29.090691 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":10964}
I20260806 07:58:29.090854 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.004s sys 0.000s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":157,"lbm_writes_lt_1ms":4,"mutex_wait_us":7493,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.090485 6648 ts_tablet_manager.cc:1916] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.091256 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.091149 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1019,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":44,"dirs.run_cpu_time_us":133,"dirs.run_wall_time_us":1428,"lbm_read_time_us":174,"lbm_reads_lt_1ms":8,"lbm_write_time_us":171,"lbm_writes_lt_1ms":4,"mutex_wait_us":4996,"reinsert_count":0,"update_count":19}
I20260806 07:58:29.091558 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.091868 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.094280 6649 ts_tablet_manager.cc:1916] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.094884 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3dc4525dae4b5f83c33b0be1ac01f6" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:29.095890 6384 leader_election.cc:343] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.096030 6384 leader_election.cc:304] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.097034 6623 raft_consensus.cc:2749] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.101032 6649 ts_tablet_manager.cc:1929] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.101171 6649 log.cc:1199] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/0ee24a9ece7f461a96e7f386b89c6208
I20260806 07:58:29.101754 6537 tablet_bootstrap.cc:654] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.099593 6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4de34f839a442818e8c8045000f7cca" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.104705 6538 leader_election.cc:290] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.106796 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2e9449a49ea445b89274f38ddca53ac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.107013 6318 raft_consensus.cc:2468] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.112108 6313 raft_consensus.cc:2468] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.118419 6537 tablet_bootstrap.cc:492] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.118625 6537 ts_tablet_manager.cc:1403] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
W20260806 07:58:29.119236 6113 leader_election.cc:343] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.119557 6537 raft_consensus.cc:359] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.119690 6779 consensus_queue.cc:1048] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.119799 6537 raft_consensus.cc:385] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.119871 6537 raft_consensus.cc:740] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.120146 6537 consensus_queue.cc:260] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.120729 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4de34f839a442818e8c8045000f7cca" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:29.120831 6537 ts_tablet_manager.cc:1434] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:29.121131 6537 tablet_bootstrap.cc:492] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.123046 6537 tablet_bootstrap.cc:654] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.124315 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.033s user 0.001s sys 0.006s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1052,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":7453,"lbm_read_time_us":231,"lbm_reads_lt_1ms":12,"lbm_write_time_us":213,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":21}
I20260806 07:58:29.127306 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":36843}
I20260806 07:58:29.127712 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":37155}
I20260806 07:58:29.128440 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:29.128777 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:29.128957 6649 ts_tablet_manager.cc:1950] T 0ee24a9ece7f461a96e7f386b89c6208 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.130316 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 0ee24a9ece7f461a96e7f386b89c6208 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
W20260806 07:58:29.131299 6113 leader_election.cc:343] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.133829 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.134979 6649 tablet_replica.cc:333] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.135205 6649 raft_consensus.cc:2243] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.135411 6649 raft_consensus.cc:2272] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 07:58:29.135900 6114 consensus_peers.cc:597] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 462655eb372b4284a2b88378bbb0f686. This is attempt 1: this message will repeat every 5th retry.
W20260806 07:58:29.136279 6114 consensus_peers.cc:597] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8961173cf8d24b8eb022e994f90970da. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:29.136539 6649 ts_tablet_manager.cc:1916] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.139928 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee24a9ece7f461a96e7f386b89c6208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:28 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.140971 6537 tablet_bootstrap.cc:492] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.141141 6537 ts_tablet_manager.cc:1403] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260806 07:58:29.143056 6537 raft_consensus.cc:359] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.143276 6537 raft_consensus.cc:385] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.143361 6537 raft_consensus.cc:740] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.143616 6537 consensus_queue.cc:260] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.144199 6314 raft_consensus.cc:2468] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.144285 6537 ts_tablet_manager.cc:1434] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 07:58:29.144619 6537 tablet_bootstrap.cc:492] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.147068 6796 raft_consensus.cc:493] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.147258 6796 raft_consensus.cc:515] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.148167 6755 ts_tablet_manager.cc:1929] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.148310 6755 log.cc:1199] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/aacb42bf87304fbe8aa1edf4cc6a9e58
I20260806 07:58:29.148291 6796 leader_election.cc:290] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.148802 6755 ts_tablet_manager.cc:1950] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.150291 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d72195b4743d194125a7b231fcc80" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.151818 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d72195b4743d194125a7b231fcc80" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.151901 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.075s user 0.005s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":208,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
W20260806 07:58:29.153952 6113 leader_election.cc:343] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.157254 6755 tablet_replica.cc:333] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.157521 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet aacb42bf87304fbe8aa1edf4cc6a9e58 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.157717 6755 raft_consensus.cc:2243] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.157979 6537 tablet_bootstrap.cc:654] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.158115 6755 raft_consensus.cc:2272] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.159209 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3036ed4cf6f04ec88c613f1bec3255d0" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.159355 6755 ts_tablet_manager.cc:1916] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.160643 6252 leader_election.cc:304] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.161379 6572 raft_consensus.cc:2749] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 07:58:29.159610 6114 leader_election.cc:343] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.161927 6114 leader_election.cc:304] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:29.162637 6796 raft_consensus.cc:2749] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.163234 6532 tablet_bootstrap.cc:492] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.163415 6532 ts_tablet_manager.cc:1403] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.161s user 0.008s sys 0.000s
I20260806 07:58:29.163579 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.073s user 0.007s sys 0.000s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":207,"lbm_writes_lt_1ms":4,"mutex_wait_us":20943,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.164346 6532 raft_consensus.cc:359] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.164805 6532 raft_consensus.cc:385] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.164914 6532 raft_consensus.cc:740] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.165248 6532 consensus_queue.cc:260] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.165948 6532 ts_tablet_manager.cc:1434] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 07:58:29.166302 6532 tablet_bootstrap.cc:492] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.168169 6532 tablet_bootstrap.cc:654] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.169808 6568 raft_consensus.cc:515] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.170544 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.007s sys 0.003s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":1062,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":63,"lbm_read_time_us":336,"lbm_reads_lt_1ms":14,"lbm_write_time_us":225,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":5}
I20260806 07:58:29.172708 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2e9449a49ea445b89274f38ddca53ac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.173038 6177 raft_consensus.cc:3060] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.170390 6649 ts_tablet_manager.cc:1929] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.174296 6649 log.cc:1199] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/7bafa7287f5b4914a9fa73cc10ac5d2b
I20260806 07:58:29.174800 6649 ts_tablet_manager.cc:1950] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.176079 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 7bafa7287f5b4914a9fa73cc10ac5d2b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.179035 6177 raft_consensus.cc:2468] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.180801 6568 leader_election.cc:290] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.180816 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2e9449a49ea445b89274f38ddca53ac" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:29.181133 6314 raft_consensus.cc:3060] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.181746 6387 leader_election.cc:304] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.182207 6568 raft_consensus.cc:2804] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 07:58:29.183724 5999 catalog_manager.cc:4995] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): delete failed for tablet 0ee24a9ece7f461a96e7f386b89c6208 because the tablet was not found. No further retry: Not found: Tablet not found: 0ee24a9ece7f461a96e7f386b89c6208
I20260806 07:58:29.184167 6568 raft_consensus.cc:697] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:29.184321 6755 ts_tablet_manager.cc:1929] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.184677 6568 consensus_queue.cc:237] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.186333 6755 log.cc:1199] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/7bafa7287f5b4914a9fa73cc10ac5d2b
I20260806 07:58:29.185062 6314 raft_consensus.cc:2468] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.187405 6755 ts_tablet_manager.cc:1950] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.188478 6531 tablet_bootstrap.cc:492] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.188692 6531 ts_tablet_manager.cc:1403] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.111s user 0.049s sys 0.058s
I20260806 07:58:29.189651 6531 raft_consensus.cc:359] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.189852 6531 raft_consensus.cc:385] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.189927 6531 raft_consensus.cc:740] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.190256 6531 consensus_queue.cc:260] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.190254 6014 catalog_manager.cc:5697] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.190742 6531 ts_tablet_manager.cc:1434] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:29.191020 6531 tablet_bootstrap.cc:492] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.192582 6531 tablet_bootstrap.cc:654] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.193326 6755 tablet_replica.cc:333] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.193768 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 7bafa7287f5b4914a9fa73cc10ac5d2b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.196458 6648 ts_tablet_manager.cc:1929] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.197224 6532 tablet_bootstrap.cc:492] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.197420 6532 ts_tablet_manager.cc:1403] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.002s
I20260806 07:58:29.198241 6532 raft_consensus.cc:359] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.198822 6648 log.cc:1199] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/aacb42bf87304fbe8aa1edf4cc6a9e58
I20260806 07:58:29.198966 6532 raft_consensus.cc:385] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.199432 6648 ts_tablet_manager.cc:1950] T aacb42bf87304fbe8aa1edf4cc6a9e58 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.199572 6532 raft_consensus.cc:740] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.199857 6532 consensus_queue.cc:260] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.201083 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet aacb42bf87304fbe8aa1edf4cc6a9e58 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.202108 6531 tablet_bootstrap.cc:492] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.202112 6532 ts_tablet_manager.cc:1434] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260806 07:58:29.202224 6537 tablet_bootstrap.cc:492] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.202270 6531 ts_tablet_manager.cc:1403] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260806 07:58:29.202411 6537 ts_tablet_manager.cc:1403] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.005s
I20260806 07:58:29.202648 6532 tablet_bootstrap.cc:492] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.202956 6648 tablet_replica.cc:333] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.203040 6531 raft_consensus.cc:359] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.203295 6531 raft_consensus.cc:385] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.203406 6531 raft_consensus.cc:740] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.203647 6531 consensus_queue.cc:260] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.204121 6755 raft_consensus.cc:2243] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.204648 6755 raft_consensus.cc:2272] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.205420 6755 ts_tablet_manager.cc:1916] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.206519 6531 ts_tablet_manager.cc:1434] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 07:58:29.203351 6537 raft_consensus.cc:359] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.206969 6537 raft_consensus.cc:385] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.207068 6537 raft_consensus.cc:740] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.207351 6537 consensus_queue.cc:260] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.207886 6537 ts_tablet_manager.cc:1434] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 07:58:29.208241 6537 tablet_bootstrap.cc:492] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.210896 6537 tablet_bootstrap.cc:654] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.211230 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.211740 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.211820 6796 raft_consensus.cc:493] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.211946 6796 raft_consensus.cc:515] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.212287 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.212561 6796 leader_election.cc:290] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.212661 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.212919 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.213263 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.213518 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39fcd48f42fc4b0a9edc04adcc8364a0" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.213553 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.213721 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.214157 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.214491 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.214813 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.215116 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.215653 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.215961 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.216224 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260806 07:58:29.217389 6114 leader_election.cc:343] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.217775 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39fcd48f42fc4b0a9edc04adcc8364a0" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.218058 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.102209
W20260806 07:58:29.218338 6113 leader_election.cc:343] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.218461 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.102209
I20260806 07:58:29.218477 6113 leader_election.cc:304] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:29.218737 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.102209
I20260806 07:58:29.218981 6796 raft_consensus.cc:2749] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.219198 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.219421 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.219741 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.220136 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.220641 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":230,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.220841 6796 consensus_queue.cc:1048] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.223614 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":777,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":63,"lbm_read_time_us":156,"lbm_reads_lt_1ms":8,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"mutex_wait_us":3674,"reinsert_count":0,"update_count":6}
I20260806 07:58:29.225881 6648 raft_consensus.cc:2243] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.226904 6796 consensus_queue.cc:1048] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.227599 6796 consensus_queue.cc:1048] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.230589 6531 tablet_bootstrap.cc:492] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.232558 6531 tablet_bootstrap.cc:654] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.233757 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.003s sys 0.002s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.235212 6531 tablet_bootstrap.cc:492] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.235392 6531 ts_tablet_manager.cc:1403] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260806 07:58:29.235904 6648 raft_consensus.cc:2272] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.236068 6531 raft_consensus.cc:359] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.236282 6531 raft_consensus.cc:385] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.236364 6531 raft_consensus.cc:740] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.236621 6531 consensus_queue.cc:260] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.237145 6531 ts_tablet_manager.cc:1434] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:29.237146 6648 ts_tablet_manager.cc:1916] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.237449 6531 tablet_bootstrap.cc:492] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.238384 6532 tablet_bootstrap.cc:654] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.239187 6531 tablet_bootstrap.cc:654] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.249336 6648 ts_tablet_manager.cc:1929] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.249657 6648 log.cc:1199] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/462655eb372b4284a2b88378bbb0f686
I20260806 07:58:29.250284 6648 ts_tablet_manager.cc:1950] T 462655eb372b4284a2b88378bbb0f686 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.251888 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 462655eb372b4284a2b88378bbb0f686 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.255806 6583 raft_consensus.cc:493] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.256042 6583 raft_consensus.cc:515] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.256930 6583 leader_election.cc:290] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.257383 6648 tablet_replica.cc:333] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.257871 6648 raft_consensus.cc:2243] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.258396 6648 raft_consensus.cc:2272] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.258898 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e15cc19276474a8b160d7959d8bf89" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.259565 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e15cc19276474a8b160d7959d8bf89" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:29.260178 6113 leader_election.cc:343] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
W20260806 07:58:29.260569 6114 consensus_peers.cc:597] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81dd62212ec041f19f9760efb21ded44. This is attempt 1: this message will repeat every 5th retry.
W20260806 07:58:29.260919 6114 leader_election.cc:343] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.261082 6114 leader_election.cc:304] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:29.261583 6531 tablet_bootstrap.cc:492] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.261606 6796 consensus_queue.cc:1048] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.261776 6531 ts_tablet_manager.cc:1403] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.010s
I20260806 07:58:29.262045 6796 raft_consensus.cc:2749] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.262686 6531 raft_consensus.cc:359] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.262912 6531 raft_consensus.cc:385] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.263023 6531 raft_consensus.cc:740] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.263366 6531 consensus_queue.cc:260] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.263895 6531 ts_tablet_manager.cc:1434] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:29.264196 6531 tablet_bootstrap.cc:492] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.266227 6532 tablet_bootstrap.cc:492] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.266404 6532 ts_tablet_manager.cc:1403] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.000s
I20260806 07:58:29.266655 6531 tablet_bootstrap.cc:654] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.267136 6648 ts_tablet_manager.cc:1916] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.267198 6532 raft_consensus.cc:359] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.267388 6532 raft_consensus.cc:385] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.267465 6532 raft_consensus.cc:740] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.267947 6532 consensus_queue.cc:260] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.270649 6532 ts_tablet_manager.cc:1434] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:29.271816 6532 tablet_bootstrap.cc:492] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.275452 6568 raft_consensus.cc:493] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.275650 6568 raft_consensus.cc:515] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.276661 6568 leader_election.cc:290] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.278041 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e15cc19276474a8b160d7959d8bf89" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.278342 6177 raft_consensus.cc:2468] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.279079 6532 tablet_bootstrap.cc:654] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.279218 6387 leader_election.cc:304] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.279579 6755 ts_tablet_manager.cc:1929] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.279731 6755 log.cc:1199] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/462655eb372b4284a2b88378bbb0f686
I20260806 07:58:29.279812 6568 raft_consensus.cc:2804] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.279959 6568 raft_consensus.cc:493] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.280072 6568 raft_consensus.cc:3060] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.280432 6755 ts_tablet_manager.cc:1950] T 462655eb372b4284a2b88378bbb0f686 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.281780 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 462655eb372b4284a2b88378bbb0f686 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.284904 6568 raft_consensus.cc:515] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.285416 6623 raft_consensus.cc:493] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.285609 6623 raft_consensus.cc:515] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.286366 6623 leader_election.cc:290] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.287889 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b87b2e51024e3d8311bbac25ae573b" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:29.288833 6387 leader_election.cc:343] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.289294 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b87b2e51024e3d8311bbac25ae573b" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.289692 6537 tablet_bootstrap.cc:492] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.289841 6755 tablet_replica.cc:333] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.289849 6537 ts_tablet_manager.cc:1403] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.082s user 0.004s sys 0.011s
I20260806 07:58:29.290066 6755 raft_consensus.cc:2243] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 07:58:29.290091 6384 leader_election.cc:343] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.290270 6755 raft_consensus.cc:2272] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.290246 6384 leader_election.cc:304] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.290772 6537 raft_consensus.cc:359] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.290948 6537 raft_consensus.cc:385] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.291046 6537 raft_consensus.cc:740] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.291330 6537 consensus_queue.cc:260] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.291708 6623 raft_consensus.cc:2749] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.293385 6648 ts_tablet_manager.cc:1929] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.293327 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e15cc19276474a8b160d7959d8bf89" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:29.294080 6384 leader_election.cc:343] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.294315 6568 leader_election.cc:290] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.293817 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e15cc19276474a8b160d7959d8bf89" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.294688 6177 raft_consensus.cc:3060] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.294675 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e15cc19276474a8b160d7959d8bf89" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:29.295447 6384 leader_election.cc:343] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.296967 6755 ts_tablet_manager.cc:1916] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.297533 6177 raft_consensus.cc:2468] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.297715 6537 ts_tablet_manager.cc:1434] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 07:58:29.298182 6387 leader_election.cc:304] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:29.298970 6568 raft_consensus.cc:2804] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.299315 6568 raft_consensus.cc:697] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:29.299712 6568 consensus_queue.cc:237] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.301038 6648 log.cc:1199] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/0ee24a9ece7f461a96e7f386b89c6208
I20260806 07:58:29.302356 6648 ts_tablet_manager.cc:1950] T 0ee24a9ece7f461a96e7f386b89c6208 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.302484 6537 tablet_bootstrap.cc:492] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.305459 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 0ee24a9ece7f461a96e7f386b89c6208 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.306107 6537 tablet_bootstrap.cc:654] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.306651 6648 tablet_replica.cc:333] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.307305 6623 raft_consensus.cc:493] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.307509 6623 raft_consensus.cc:515] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.307837 6648 raft_consensus.cc:2243] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.308238 6648 raft_consensus.cc:2272] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.308497 6623 leader_election.cc:290] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.309247 6648 ts_tablet_manager.cc:1916] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.309981 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:29.310637 6387 leader_election.cc:343] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.312070 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.312350 6572 raft_consensus.cc:493] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.312536 6572 raft_consensus.cc:515] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.313148 6572 leader_election.cc:290] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.313777 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.095s user 0.016s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":14541,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":216,"lbm_read_time_us":530,"lbm_reads_lt_1ms":23,"lbm_write_time_us":450,"lbm_writes_lt_1ms":9,"mutex_wait_us":2772,"peak_mem_usage":32768,"reinsert_count":0,"update_count":22}
I20260806 07:58:29.314898 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4bee6c27bf4e9198f13f81ea27678e" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.315294 6177 raft_consensus.cc:2393] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
W20260806 07:58:29.318464 6384 leader_election.cc:343] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.318552 6532 tablet_bootstrap.cc:492] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.318632 6384 leader_election.cc:304] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.318718 6532 ts_tablet_manager.cc:1403] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.000s
I20260806 07:58:29.319620 6532 raft_consensus.cc:359] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.319706 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4bee6c27bf4e9198f13f81ea27678e" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.319829 6532 raft_consensus.cc:385] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.319922 6532 raft_consensus.cc:740] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.320210 6532 consensus_queue.cc:260] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.320511 6250 leader_election.cc:304] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.320853 6532 ts_tablet_manager.cc:1434] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:29.321055 6538 raft_consensus.cc:3060] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.321203 6532 tablet_bootstrap.cc:492] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.323053 6532 tablet_bootstrap.cc:654] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.324829 6648 ts_tablet_manager.cc:1929] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.325512 6538 raft_consensus.cc:2749] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.326332 6648 log.cc:1199] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/06cc4788571140b2ab78811966d2e06d
I20260806 07:58:29.326817 6537 tablet_bootstrap.cc:492] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.327025 6537 ts_tablet_manager.cc:1403] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.000s
I20260806 07:58:29.327682 6648 ts_tablet_manager.cc:1950] T 06cc4788571140b2ab78811966d2e06d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.328740 6623 raft_consensus.cc:2749] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.327889 6537 raft_consensus.cc:359] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.329075 6537 raft_consensus.cc:385] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.329171 6537 raft_consensus.cc:740] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.329311 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 06cc4788571140b2ab78811966d2e06d (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.329478 6537 consensus_queue.cc:260] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.330787 6648 tablet_replica.cc:333] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.331357 6648 raft_consensus.cc:2243] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.331894 6648 raft_consensus.cc:2272] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.331318 6755 ts_tablet_manager.cc:1929] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.333170 6755 log.cc:1199] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/81dd62212ec041f19f9760efb21ded44
I20260806 07:58:29.333204 6648 ts_tablet_manager.cc:1916] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.333774 6755 ts_tablet_manager.cc:1950] T 81dd62212ec041f19f9760efb21ded44 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.334386 6531 tablet_bootstrap.cc:492] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.334642 6531 ts_tablet_manager.cc:1403] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.070s user 0.032s sys 0.022s
I20260806 07:58:29.335472 6531 raft_consensus.cc:359] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.335722 6531 raft_consensus.cc:385] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.335822 6531 raft_consensus.cc:740] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.335807 6623 consensus_queue.cc:1048] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.336191 6531 consensus_queue.cc:260] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.336753 6531 ts_tablet_manager.cc:1434] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:29.337055 6531 tablet_bootstrap.cc:492] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.333220 6537 ts_tablet_manager.cc:1434] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260806 07:58:29.340545 6537 tablet_bootstrap.cc:492] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.341573 6648 ts_tablet_manager.cc:1929] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.341820 6648 log.cc:1199] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/7bafa7287f5b4914a9fa73cc10ac5d2b
I20260806 07:58:29.342362 6648 ts_tablet_manager.cc:1950] T 7bafa7287f5b4914a9fa73cc10ac5d2b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.343554 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 7bafa7287f5b4914a9fa73cc10ac5d2b (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.343927 6648 tablet_replica.cc:333] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.344163 6537 tablet_bootstrap.cc:654] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.344174 6648 raft_consensus.cc:2243] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.344594 6648 raft_consensus.cc:2272] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.345649 6648 ts_tablet_manager.cc:1916] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.345897 6014 catalog_manager.cc:5697] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.350160 6531 tablet_bootstrap.cc:654] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.351225 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 81dd62212ec041f19f9760efb21ded44 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.352636 6755 tablet_replica.cc:333] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.352911 6755 raft_consensus.cc:2243] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.353300 6538 raft_consensus.cc:493] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.353348 6622 raft_consensus.cc:493] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.353555 6622 raft_consensus.cc:515] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.354748 6622 leader_election.cc:290] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.356195 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f054c628591a4ef7ab8d607abd77a984" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.356472 6177 raft_consensus.cc:2468] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.357211 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.357679 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.357795 6387 leader_election.cc:304] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.357985 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.358259 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.358409 6568 raft_consensus.cc:2804] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.358525 6568 raft_consensus.cc:493] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.358570 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.358628 6568 raft_consensus.cc:3060] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.358950 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.359247 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.359539 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.360124 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.360385 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.360594 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.361826 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.362188 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.362495 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.363711 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.364035 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.364305 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.364653 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.364951 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.365353 6622 consensus_queue.cc:1048] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.365300 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":193,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.361513 6568 raft_consensus.cc:515] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.365927 6538 raft_consensus.cc:515] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.366714 6538 leader_election.cc:290] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.367499 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d72195b4743d194125a7b231fcc80" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.367702 6450 raft_consensus.cc:2468] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:29.369480 6250 leader_election.cc:304] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:29.370172 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d72195b4743d194125a7b231fcc80" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.370209 6572 raft_consensus.cc:2804] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.370339 6572 raft_consensus.cc:493] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.370375 6177 raft_consensus.cc:2468] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:29.370464 6572 raft_consensus.cc:3060] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.375058 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f054c628591a4ef7ab8d607abd77a984" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.375243 6177 raft_consensus.cc:3060] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.378268 6177 raft_consensus.cc:2468] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.378859 6387 leader_election.cc:304] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.379246 6623 raft_consensus.cc:2804] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.379935 6623 raft_consensus.cc:697] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:29.380306 6623 consensus_queue.cc:237] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.380939 6532 tablet_bootstrap.cc:492] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.381148 6532 ts_tablet_manager.cc:1403] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.060s user 0.002s sys 0.002s
I20260806 07:58:29.382087 6532 raft_consensus.cc:359] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.382313 6532 raft_consensus.cc:385] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.382385 6532 raft_consensus.cc:740] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.382876 6532 consensus_queue.cc:260] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.383404 6532 ts_tablet_manager.cc:1434] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:29.383531 6755 raft_consensus.cc:2272] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.383737 6568 leader_election.cc:290] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.384266 6532 tablet_bootstrap.cc:492] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.386535 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f054c628591a4ef7ab8d607abd77a984" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:29.387912 6384 leader_election.cc:343] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.392741 6572 raft_consensus.cc:515] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.393198 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f054c628591a4ef7ab8d607abd77a984" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.395243 6532 tablet_bootstrap.cc:654] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:29.395737 6384 leader_election.cc:343] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.397257 6755 ts_tablet_manager.cc:1916] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.397636 6537 tablet_bootstrap.cc:492] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.397814 6537 ts_tablet_manager.cc:1403] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.057s user 0.004s sys 0.001s
I20260806 07:58:29.398723 6537 raft_consensus.cc:359] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.398923 6537 raft_consensus.cc:385] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.399011 6537 raft_consensus.cc:740] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.399334 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d72195b4743d194125a7b231fcc80" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.399546 6177 raft_consensus.cc:3060] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.400452 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d72195b4743d194125a7b231fcc80" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.400552 6572 leader_election.cc:290] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.400660 6450 raft_consensus.cc:3060] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.402171 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.042s user 0.002s sys 0.005s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":205,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.402979 6177 raft_consensus.cc:2468] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:29.403517 6252 leader_election.cc:304] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.403899 6450 raft_consensus.cc:2468] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:29.403996 6572 raft_consensus.cc:2804] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.404199 6538 raft_consensus.cc:493] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.404317 6538 raft_consensus.cc:515] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.404937 6538 leader_election.cc:290] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.406325 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3dc4525dae4b5f83c33b0be1ac01f6" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.406594 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3dc4525dae4b5f83c33b0be1ac01f6" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.406613 6450 raft_consensus.cc:2468] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:29.406795 6177 raft_consensus.cc:2468] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:29.407219 6250 leader_election.cc:304] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:29.407498 6572 raft_consensus.cc:697] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:29.407608 6592 raft_consensus.cc:2804] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.407757 6592 raft_consensus.cc:493] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.407873 6592 raft_consensus.cc:3060] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.407819 6572 consensus_queue.cc:237] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.411253 6592 raft_consensus.cc:515] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.411473 6648 ts_tablet_manager.cc:1929] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.411620 6648 log.cc:1199] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/8961173cf8d24b8eb022e994f90970da
I20260806 07:58:29.411962 6592 leader_election.cc:290] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.412791 6537 consensus_queue.cc:260] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.413007 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3dc4525dae4b5f83c33b0be1ac01f6" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.413247 6453 raft_consensus.cc:3060] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.413411 6537 ts_tablet_manager.cc:1434] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.015s user 0.000s sys 0.002s
I20260806 07:58:29.413762 6537 tablet_bootstrap.cc:492] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.412744 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.004s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":806,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":6114,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":724,"lbm_read_time_us":252,"lbm_reads_lt_1ms":8,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"mutex_wait_us":16757,"reinsert_count":0,"update_count":7}
I20260806 07:58:29.418239 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3dc4525dae4b5f83c33b0be1ac01f6" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.418545 6177 raft_consensus.cc:3060] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.420644 6648 ts_tablet_manager.cc:1950] T 8961173cf8d24b8eb022e994f90970da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.422300 6648 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39230 (request call id 58) took 1036 ms. Trace:
I20260806 07:58:29.422472 6648 rpcz_store.cc:276] 0806 07:58:28.386114 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:28.387495 (+ 1381us) service_pool.cc:224] Handling call
0806 07:58:29.343848 (+956353us) ts_tablet_manager.cc:1151] Deleting tablet 8961173cf8d24b8eb022e994f90970da
0806 07:58:29.343856 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:29.351070 (+ 7214us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.411348 (+ 60278us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.422245 (+ 10897us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":956181}
I20260806 07:58:29.423202 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 8961173cf8d24b8eb022e994f90970da (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.423713 6648 tablet_replica.cc:333] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.424081 6648 raft_consensus.cc:2243] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.425175 6648 raft_consensus.cc:2272] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.425232 6453 raft_consensus.cc:2468] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:29.425879 6250 leader_election.cc:304] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:29.426327 6718 raft_consensus.cc:2804] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.426543 6718 raft_consensus.cc:697] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:29.426913 6718 consensus_queue.cc:237] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.428936 6755 ts_tablet_manager.cc:1929] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.433632 6177 raft_consensus.cc:2468] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:29.436360 6648 ts_tablet_manager.cc:1916] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.438221 6531 tablet_bootstrap.cc:492] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.438355 6531 ts_tablet_manager.cc:1403] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.101s user 0.020s sys 0.040s
I20260806 07:58:29.439285 6531 raft_consensus.cc:359] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.439492 6531 raft_consensus.cc:385] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.439586 6531 raft_consensus.cc:740] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.439910 6755 log.cc:1199] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/8961173cf8d24b8eb022e994f90970da
I20260806 07:58:29.439888 6531 consensus_queue.cc:260] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:29.440056 6252 consensus_peers.cc:597] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 -> Peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Couldn't send request to peer 7793159fc1ab4bd1b2a50dbd3b3e5168. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06cc4788571140b2ab78811966d2e06d. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:29.439893 6014 catalog_manager.cc:5697] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.440398 6531 ts_tablet_manager.cc:1434] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:29.440423 6755 ts_tablet_manager.cc:1950] T 8961173cf8d24b8eb022e994f90970da P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.441342 6755 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36458 (request call id 58) took 1048 ms. Trace:
I20260806 07:58:29.442026 6531 tablet_bootstrap.cc:492] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.444072 6531 tablet_bootstrap.cc:654] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.445120 6015 catalog_manager.cc:5697] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.447587 6537 tablet_bootstrap.cc:654] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.441504 6755 rpcz_store.cc:276] 0806 07:58:28.392669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:28.393908 (+ 1239us) service_pool.cc:224] Handling call
0806 07:58:29.352520 (+958612us) ts_tablet_manager.cc:1151] Deleting tablet 8961173cf8d24b8eb022e994f90970da
0806 07:58:29.352533 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:29.402948 (+ 50415us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.428834 (+ 25886us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.441292 (+ 12458us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":958458}
I20260806 07:58:29.450698 6532 tablet_bootstrap.cc:492] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.457813 6532 ts_tablet_manager.cc:1403] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.074s user 0.004s sys 0.004s
I20260806 07:58:29.458766 6532 raft_consensus.cc:359] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.459046 6532 raft_consensus.cc:385] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.459179 6532 raft_consensus.cc:740] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.459517 6532 consensus_queue.cc:260] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.460222 6532 ts_tablet_manager.cc:1434] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:29.460585 6532 tablet_bootstrap.cc:492] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.460773 6648 ts_tablet_manager.cc:1929] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.460934 6648 log.cc:1199] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/81dd62212ec041f19f9760efb21ded44
I20260806 07:58:29.461412 6648 ts_tablet_manager.cc:1950] T 81dd62212ec041f19f9760efb21ded44 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.461941 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.462378 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.464109 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.464500 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.464807 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.465080 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.465384 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.465641 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.465878 6537 tablet_bootstrap.cc:492] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.466053 6537 ts_tablet_manager.cc:1403] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.000s
I20260806 07:58:29.466140 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.466499 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.466871 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.466931 6537 raft_consensus.cc:359] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.467126 6537 raft_consensus.cc:385] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.467156 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.467203 6537 raft_consensus.cc:740] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.467454 6537 consensus_queue.cc:260] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.467974 6537 ts_tablet_manager.cc:1434] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:29.468318 6537 tablet_bootstrap.cc:492] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:29.469000 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.469280 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.469514 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.471058 6537 tablet_bootstrap.cc:654] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.471097 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.471360 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.471609 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.471907 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.472142 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.476684 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.476960 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.477674 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.015s user 0.001s sys 0.004s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":144,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.478216 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.478461 6648 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39230 (request call id 63) took 1086 ms. Trace:
I20260806 07:58:29.478576 6648 rpcz_store.cc:276] 0806 07:58:28.391978 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:28.392041 (+ 63us) service_pool.cc:224] Handling call
0806 07:58:29.423648 (+1031607us) ts_tablet_manager.cc:1151] Deleting tablet 81dd62212ec041f19f9760efb21ded44
0806 07:58:29.423658 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:29.452891 (+ 29233us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.460686 (+ 7795us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.478387 (+ 17701us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1031393}
I20260806 07:58:29.479084 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 81dd62212ec041f19f9760efb21ded44 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.479701 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":999,"lbm_read_time_us":170,"lbm_reads_lt_1ms":8,"lbm_write_time_us":157,"lbm_writes_lt_1ms":4,"mutex_wait_us":4875,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.480477 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.480324 6015 catalog_manager.cc:5697] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.480733 6531 tablet_bootstrap.cc:492] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.480924 6531 ts_tablet_manager.cc:1403] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.029s
I20260806 07:58:29.481202 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 8961173cf8d24b8eb022e994f90970da (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.481621 6755 tablet_replica.cc:333] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.481873 6755 raft_consensus.cc:2243] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.481815 6531 raft_consensus.cc:359] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.482028 6531 raft_consensus.cc:385] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.482079 6755 raft_consensus.cc:2272] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.482105 6531 raft_consensus.cc:740] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.482359 6531 consensus_queue.cc:260] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.482852 6531 ts_tablet_manager.cc:1434] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:29.484344 6531 tablet_bootstrap.cc:492] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.484905 6532 tablet_bootstrap.cc:654] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.486884 6755 ts_tablet_manager.cc:1916] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.487191 6568 consensus_queue.cc:1048] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.488617 6531 tablet_bootstrap.cc:654] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.490255 6568 raft_consensus.cc:493] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.490455 6568 raft_consensus.cc:515] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.491200 6568 leader_election.cc:290] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.491902 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1e9ae4d73d46d99f50b701d63d4f18" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.492197 6177 raft_consensus.cc:2468] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.492806 6387 leader_election.cc:304] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.493716 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1e9ae4d73d46d99f50b701d63d4f18" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:29.494362 6384 leader_election.cc:343] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.495048 6568 raft_consensus.cc:2804] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.495149 6568 raft_consensus.cc:493] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.495262 6568 raft_consensus.cc:3060] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.498008 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.498375 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.499754 6833 consensus_queue.cc:1048] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.505043 6583 consensus_queue.cc:1048] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.505549 6537 tablet_bootstrap.cc:492] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:29.481482 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.004s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":185,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.505728 6537 ts_tablet_manager.cc:1403] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.000s
I20260806 07:58:29.505940 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.506511 6537 raft_consensus.cc:359] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.506683 6537 raft_consensus.cc:385] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.506758 6537 raft_consensus.cc:740] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.506975 6537 consensus_queue.cc:260] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.507475 6537 ts_tablet_manager.cc:1434] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:29.514505 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":15726}
I20260806 07:58:29.515394 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":9153}
I20260806 07:58:29.515913 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.006s sys 0.000s Metrics: {"bytes_written":464,"delete_count":0,"lbm_write_time_us":166,"lbm_writes_lt_1ms":4,"mutex_wait_us":9900,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.516501 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.045s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":836,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":605,"lbm_read_time_us":232,"lbm_reads_lt_1ms":8,"lbm_write_time_us":197,"lbm_writes_lt_1ms":4,"mutex_wait_us":799,"reinsert_count":0,"update_count":8}
I20260806 07:58:29.517218 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.517531 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.517421 6568 raft_consensus.cc:515] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.517815 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.518060 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.518911 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1e9ae4d73d46d99f50b701d63d4f18" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.519150 6179 raft_consensus.cc:3060] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.520458 6755 ts_tablet_manager.cc:1929] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.520658 6755 log.cc:1199] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/0ee24a9ece7f461a96e7f386b89c6208
I20260806 07:58:29.522370 6179 raft_consensus.cc:2468] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.523002 6387 leader_election.cc:304] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.523484 6833 raft_consensus.cc:2804] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.523677 6833 raft_consensus.cc:697] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:29.524012 6833 consensus_queue.cc:237] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.525432 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":883,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":52,"lbm_read_time_us":169,"lbm_reads_lt_1ms":8,"lbm_write_time_us":163,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":10}
I20260806 07:58:29.526252 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.526530 6583 raft_consensus.cc:493] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.526750 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.526695 6583 raft_consensus.cc:515] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.527123 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.527541 6583 leader_election.cc:290] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.528352 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d68050d52894d868b86370cd5e449b8" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:29.529068 6113 leader_election.cc:343] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.529520 6755 ts_tablet_manager.cc:1950] T 0ee24a9ece7f461a96e7f386b89c6208 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.529933 6531 tablet_bootstrap.cc:492] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:29.530110 6531 ts_tablet_manager.cc:1403] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.046s user 0.013s sys 0.013s
I20260806 07:58:29.530484 6755 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36458 (request call id 60) took 1137 ms. Trace:
I20260806 07:58:29.530977 6531 raft_consensus.cc:359] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.531155 6531 raft_consensus.cc:385] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.531250 6531 raft_consensus.cc:740] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.531486 6531 consensus_queue.cc:260] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.532171 6531 ts_tablet_manager.cc:1434] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:29.530580 6755 rpcz_store.cc:276] 0806 07:58:28.392890 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:28.394082 (+ 1192us) service_pool.cc:224] Handling call
0806 07:58:29.481524 (+1087442us) ts_tablet_manager.cc:1151] Deleting tablet 0ee24a9ece7f461a96e7f386b89c6208
0806 07:58:29.481537 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:29.497956 (+ 16419us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.520343 (+ 22387us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.530418 (+ 10075us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1087308}
I20260806 07:58:29.532627 6755 tablet_replica.cc:333] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.532819 6755 raft_consensus.cc:2243] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.533226 6755 raft_consensus.cc:2272] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.534472 6572 raft_consensus.cc:493] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.534495 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d68050d52894d868b86370cd5e449b8" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.534687 6453 raft_consensus.cc:2468] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:29.534658 6572 raft_consensus.cc:515] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.535883 6572 leader_election.cc:290] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.536322 6114 leader_election.cc:304] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:29.536705 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4de34f839a442818e8c8045000f7cca" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.537431 6583 raft_consensus.cc:2804] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.537637 6583 raft_consensus.cc:493] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.537748 6583 raft_consensus.cc:3060] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.538455 6755 ts_tablet_manager.cc:1916] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.540802 6583 raft_consensus.cc:515] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.541689 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 0ee24a9ece7f461a96e7f386b89c6208 (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.543298 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d68050d52894d868b86370cd5e449b8" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:29.543912 6113 leader_election.cc:343] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.544119 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d68050d52894d868b86370cd5e449b8" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.544349 6453 raft_consensus.cc:3060] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.545884 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4de34f839a442818e8c8045000f7cca" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.546708 6252 leader_election.cc:304] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.547152 6572 raft_consensus.cc:2749] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.551240 6583 leader_election.cc:290] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.553090 6568 leader_election.cc:290] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.554580 6453 raft_consensus.cc:2468] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.556744 6755 ts_tablet_manager.cc:1929] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.556748 6114 leader_election.cc:304] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:29.556886 6755 log.cc:1199] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/06cc4788571140b2ab78811966d2e06d
I20260806 07:58:29.557406 6755 ts_tablet_manager.cc:1950] T 06cc4788571140b2ab78811966d2e06d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.560286 6755 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36458 (request call id 62) took 1167 ms. Trace:
I20260806 07:58:29.560422 6755 rpcz_store.cc:276] 0806 07:58:28.393065 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:28.394232 (+ 1167us) service_pool.cc:224] Handling call
0806 07:58:29.532539 (+1138307us) ts_tablet_manager.cc:1151] Deleting tablet 06cc4788571140b2ab78811966d2e06d
0806 07:58:29.532550 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:29.546911 (+ 14361us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.556635 (+ 9724us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:29.560234 (+ 3599us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1138147}
I20260806 07:58:29.560933 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 06cc4788571140b2ab78811966d2e06d (table table-1 [id=d056973627284d368d205bef9293abd6]) successfully deleted
I20260806 07:58:29.563970 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":64569}
I20260806 07:58:29.564836 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.000s sys 0.003s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"mutex_wait_us":52828,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.569646 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.057s user 0.004s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"mutex_wait_us":2483,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.570037 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1e9ae4d73d46d99f50b701d63d4f18" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:29.571313 6384 leader_election.cc:343] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.571959 6583 raft_consensus.cc:2804] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.572345 6583 raft_consensus.cc:697] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:29.577247 6532 tablet_bootstrap.cc:492] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.577462 6532 ts_tablet_manager.cc:1403] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.117s user 0.011s sys 0.023s
I20260806 07:58:29.578394 6532 raft_consensus.cc:359] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.578603 6532 raft_consensus.cc:385] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.578423 6014 catalog_manager.cc:5697] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.578708 6532 raft_consensus.cc:740] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.579017 6532 consensus_queue.cc:260] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.579589 6532 ts_tablet_manager.cc:1434] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 07:58:29.579952 6532 tablet_bootstrap.cc:492] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.582026 6532 tablet_bootstrap.cc:654] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.588512 6312 raft_consensus.cc:3060] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.591614 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.591922 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.592206 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.598311 6583 consensus_queue.cc:237] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.600740 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.601094 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.601398 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.601675 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.603639 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.603914 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.604169 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.608539 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.004s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":169,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.609056 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.609351 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.609208 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.004s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"mutex_wait_us":11383,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.609845 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":11731}
I20260806 07:58:29.610248 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":12035}
I20260806 07:58:29.610634 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.130s user 0.010s sys 0.000s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":1044,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":34296,"dirs.run_cpu_time_us":163,"dirs.run_wall_time_us":2022,"lbm_read_time_us":366,"lbm_reads_lt_1ms":14,"lbm_write_time_us":215,"lbm_writes_lt_1ms":4,"mutex_wait_us":62362,"reinsert_count":0,"update_count":4}
I20260806 07:58:29.611160 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.611428 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.611688 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.611959 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.616135 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.616458 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.616811 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.617345 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.618045 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.618294 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.619244 6568 consensus_queue.cc:1048] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.626360 6779 raft_consensus.cc:493] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.626549 6779 raft_consensus.cc:515] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.627261 6779 leader_election.cc:290] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.628041 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.628278 6450 raft_consensus.cc:2468] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:29.628221 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.628779 6114 leader_election.cc:304] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
W20260806 07:58:29.628942 6113 leader_election.cc:343] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.629130 6779 raft_consensus.cc:2804] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.629246 6779 raft_consensus.cc:493] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.629334 6779 raft_consensus.cc:3060] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.632584 6779 raft_consensus.cc:515] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.633380 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.024s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":915,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":59,"lbm_read_time_us":182,"lbm_reads_lt_1ms":8,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":11}
I20260806 07:58:29.633900 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:29.634548 6113 leader_election.cc:343] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:29.634922 6779 leader_election.cc:290] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.635742 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.635968 6450 raft_consensus.cc:3060] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.636986 6779 consensus_queue.cc:1048] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.639400 6450 raft_consensus.cc:2468] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.642158 6114 leader_election.cc:304] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:29.642627 6779 raft_consensus.cc:2804] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.642804 6779 raft_consensus.cc:697] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:29.643126 6779 consensus_queue.cc:237] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.650955 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.039s user 0.001s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":758,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":77,"lbm_read_time_us":160,"lbm_reads_lt_1ms":8,"lbm_write_time_us":154,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":5}
I20260806 07:58:29.656540 6622 consensus_queue.cc:1048] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.658623 6014 catalog_manager.cc:5697] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.659282 6014 catalog_manager.cc:5697] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.663018 6532 tablet_bootstrap.cc:492] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.663180 6532 ts_tablet_manager.cc:1403] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.083s user 0.033s sys 0.041s
I20260806 07:58:29.665625 6532 raft_consensus.cc:359] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.665830 6532 raft_consensus.cc:385] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.665927 6532 raft_consensus.cc:740] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.666142 6532 consensus_queue.cc:260] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.666811 6532 ts_tablet_manager.cc:1434] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 07:58:29.667114 6532 tablet_bootstrap.cc:492] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.668972 6532 tablet_bootstrap.cc:654] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.671118 6779 raft_consensus.cc:493] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.671355 6779 raft_consensus.cc:515] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.672012 6532 tablet_bootstrap.cc:492] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.672175 6532 ts_tablet_manager.cc:1403] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 07:58:29.672305 6779 leader_election.cc:290] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.672650 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b87b2e51024e3d8311bbac25ae573b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.672895 6318 raft_consensus.cc:2468] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:29.673045 6532 raft_consensus.cc:359] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.673142 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b87b2e51024e3d8311bbac25ae573b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.673249 6532 raft_consensus.cc:385] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.673317 6532 raft_consensus.cc:740] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.673357 6450 raft_consensus.cc:2468] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:29.673348 6113 leader_election.cc:304] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.673691 6779 raft_consensus.cc:2804] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.673650 6532 consensus_queue.cc:260] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.673935 6779 raft_consensus.cc:493] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.674098 6779 raft_consensus.cc:3060] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.674149 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.674492 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.674762 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.675405 6532 ts_tablet_manager.cc:1434] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 07:58:29.676093 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.676326 6532 tablet_bootstrap.cc:492] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.676442 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.676549 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.676744 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.676858 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.677053 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.677320 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.117575
I20260806 07:58:29.677654 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.117575
I20260806 07:58:29.677824 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.677937 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.117575
I20260806 07:58:29.678155 6532 tablet_bootstrap.cc:654] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.678130 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.678288 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.678372 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.678545 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.678897 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.679358 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.683169 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.002s sys 0.002s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":198,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.684135 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.684508 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.684798 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.685127 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.688863 6779 raft_consensus.cc:515] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.689055 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.689311 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.689539 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.690590 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b87b2e51024e3d8311bbac25ae573b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:29.690820 6318 raft_consensus.cc:3060] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.691213 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b87b2e51024e3d8311bbac25ae573b" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.691493 6450 raft_consensus.cc:3060] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.693990 6318 raft_consensus.cc:2468] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.694530 6113 leader_election.cc:304] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.694911 6583 raft_consensus.cc:2804] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.695108 6583 raft_consensus.cc:697] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:29.695406 6583 consensus_queue.cc:237] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.695551 6450 raft_consensus.cc:2468] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.697101 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":955,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":61,"lbm_read_time_us":188,"lbm_reads_lt_1ms":8,"lbm_write_time_us":165,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":13}
I20260806 07:58:29.698093 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.005s sys 0.000s Metrics: {"bytes_written":459,"delete_count":0,"lbm_write_time_us":181,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.698721 6779 leader_election.cc:290] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.700224 6532 tablet_bootstrap.cc:492] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.700388 6532 ts_tablet_manager.cc:1403] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.011s
I20260806 07:58:29.702113 6318 raft_consensus.cc:3060] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.703393 6532 raft_consensus.cc:359] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.704267 6532 raft_consensus.cc:385] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.704384 6532 raft_consensus.cc:740] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.704628 6532 consensus_queue.cc:260] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.704820 6015 catalog_manager.cc:5697] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.705852 6532 ts_tablet_manager.cc:1434] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260806 07:58:29.706159 6532 tablet_bootstrap.cc:492] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.706594 6833 consensus_queue.cc:1048] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.708048 6532 tablet_bootstrap.cc:654] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.713318 6622 consensus_queue.cc:1048] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.714121 6532 tablet_bootstrap.cc:492] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.714275 6532 ts_tablet_manager.cc:1403] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260806 07:58:29.714434 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.010s sys 0.003s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":18289,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":99,"lbm_read_time_us":632,"lbm_reads_lt_1ms":27,"lbm_write_time_us":463,"lbm_writes_lt_1ms":9,"mutex_wait_us":5159,"peak_mem_usage":32768,"reinsert_count":0,"update_count":7}
I20260806 07:58:29.715332 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.715200 6532 raft_consensus.cc:359] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.715497 6532 raft_consensus.cc:385] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.715588 6532 raft_consensus.cc:740] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.715659 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.715867 6532 consensus_queue.cc:260] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.715989 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.716367 6532 ts_tablet_manager.cc:1434] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:29.716732 6532 tablet_bootstrap.cc:492] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:29.716845 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.717411 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.718899 6532 tablet_bootstrap.cc:654] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.723603 6532 tablet_bootstrap.cc:492] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:29.723781 6532 ts_tablet_manager.cc:1403] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260806 07:58:29.724565 6532 raft_consensus.cc:359] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.724774 6532 raft_consensus.cc:385] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:29.724717 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.002s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":155,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.724874 6532 raft_consensus.cc:740] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:29.725111 6532 consensus_queue.cc:260] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.725735 6532 ts_tablet_manager.cc:1434] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:29.728868 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.729260 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.729719 6622 raft_consensus.cc:493] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.729916 6622 raft_consensus.cc:515] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.730697 6622 leader_election.cc:290] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.730928 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.731122 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39fcd48f42fc4b0a9edc04adcc8364a0" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.731383 6177 raft_consensus.cc:2468] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.731411 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.731366 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39fcd48f42fc4b0a9edc04adcc8364a0" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.731613 6312 raft_consensus.cc:2468] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:29.731717 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.731791 6387 leader_election.cc:304] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.731992 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.732309 6622 raft_consensus.cc:2804] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.732295 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.732410 6622 raft_consensus.cc:493] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.732540 6622 raft_consensus.cc:3060] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.732613 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.735718 6622 raft_consensus.cc:515] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.736841 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39fcd48f42fc4b0a9edc04adcc8364a0" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:29.736901 6622 leader_election.cc:290] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:29.736986 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39fcd48f42fc4b0a9edc04adcc8364a0" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:29.737083 6177 raft_consensus.cc:3060] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.737195 6312 raft_consensus.cc:3060] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.738932 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.740247 6177 raft_consensus.cc:2468] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.740249 6312 raft_consensus.cc:2468] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:29.740777 6384 leader_election.cc:304] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:29.741075 6622 raft_consensus.cc:2804] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.741281 6622 raft_consensus.cc:697] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:29.741896 6622 consensus_queue.cc:237] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.742852 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":983,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":109,"lbm_read_time_us":136,"lbm_reads_lt_1ms":8,"lbm_write_time_us":136,"lbm_writes_lt_1ms":4,"mutex_wait_us":4057,"reinsert_count":0,"update_count":14}
I20260806 07:58:29.745549 6014 catalog_manager.cc:5697] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:29.753862 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.754232 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.754514 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.754896 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.756279 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.756703 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.757113 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.757496 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.758751 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.003s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":170,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.760823 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":165,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.765185 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1007,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":71,"lbm_read_time_us":103,"lbm_reads_lt_1ms":8,"lbm_write_time_us":190,"lbm_writes_lt_1ms":4,"mutex_wait_us":3617,"reinsert_count":0,"update_count":15}
I20260806 07:58:29.766992 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.001s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":132,"lbm_read_time_us":179,"lbm_reads_lt_1ms":8,"lbm_write_time_us":209,"lbm_writes_lt_1ms":4,"mutex_wait_us":3070,"reinsert_count":0,"update_count":2}
I20260806 07:58:29.769932 6572 raft_consensus.cc:493] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.770169 6572 raft_consensus.cc:515] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.770991 6572 leader_election.cc:290] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.771678 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f054c628591a4ef7ab8d607abd77a984" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.771682 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f054c628591a4ef7ab8d607abd77a984" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.771941 6177 raft_consensus.cc:2393] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:29.772449 6250 leader_election.cc:304] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.772727 6572 raft_consensus.cc:3060] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.774713 6572 raft_consensus.cc:2749] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.778379 6583 raft_consensus.cc:493] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.778573 6583 raft_consensus.cc:515] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.779255 6583 leader_election.cc:290] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.779783 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fe3aedfb004f9185b6307e0c087b63" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:29.779805 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fe3aedfb004f9185b6307e0c087b63" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.780005 6312 raft_consensus.cc:2468] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:29.780068 6450 raft_consensus.cc:2468] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:29.780494 6114 leader_election.cc:304] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.780791 6583 raft_consensus.cc:2804] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:29.780929 6583 raft_consensus.cc:493] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:29.781096 6583 raft_consensus.cc:3060] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.783116 6583 raft_consensus.cc:515] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.783730 6583 leader_election.cc:290] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.784216 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fe3aedfb004f9185b6307e0c087b63" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:29.784297 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fe3aedfb004f9185b6307e0c087b63" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:29.784431 6312 raft_consensus.cc:3060] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.784538 6450 raft_consensus.cc:3060] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.787313 6312 raft_consensus.cc:2468] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.787784 6113 leader_election.cc:304] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:29.788259 6779 raft_consensus.cc:2804] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:29.788407 6779 raft_consensus.cc:697] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:29.788682 6779 consensus_queue.cc:237] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.789515 6450 raft_consensus.cc:2468] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:29.791819 6014 catalog_manager.cc:5697] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:29.798895 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.799371 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.799773 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.800148 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.803150 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":171,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.807509 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.808135 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.808115 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.002s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":711,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":59,"lbm_read_time_us":101,"lbm_reads_lt_1ms":8,"lbm_write_time_us":123,"lbm_writes_lt_1ms":4,"mutex_wait_us":2804,"reinsert_count":0,"update_count":3}
I20260806 07:58:29.808537 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.808943 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:29.811718 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.004s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":231,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:29.817320 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":1029,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":88,"lbm_read_time_us":117,"lbm_reads_lt_1ms":8,"lbm_write_time_us":160,"lbm_writes_lt_1ms":4,"mutex_wait_us":2540,"reinsert_count":0,"update_count":16}
I20260806 07:58:29.827728 6014 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-2" } modify_external_catalogs: true
I20260806 07:58:29.828001 6014 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-2" } modify_external_catalogs: true
I20260806 07:58:29.839124 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.839520 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.839825 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.839898 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.840204 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.840356 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.840497 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.840509 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.840812 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.840801 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.842192 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.842483 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.842703 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.843087 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.413847
I20260806 07:58:29.843560 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.413847
I20260806 07:58:29.843838 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.844070 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:29.844739 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.845727 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet a30189879fbf49c796eea2fe621b9d9a
I20260806 07:58:29.846498 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet a30189879fbf49c796eea2fe621b9d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.846820 6648 tablet_replica.cc:333] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.847040 6648 raft_consensus.cc:2243] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.847101 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.847415 6648 raft_consensus.cc:2272] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.847532 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.847976 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet a30189879fbf49c796eea2fe621b9d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.848315 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet a30189879fbf49c796eea2fe621b9d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.848353 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.002s sys 0.003s Metrics: {"bytes_written":801,"delete_count":0,"lbm_write_time_us":202,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":12}
I20260806 07:58:29.848467 6648 ts_tablet_manager.cc:1916] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.848791 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.851011 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.000s Metrics: {"bytes_written":740,"delete_count":0,"lbm_write_time_us":186,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":2241536,"update_count":9}
I20260806 07:58:29.853811 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet a30189879fbf49c796eea2fe621b9d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.853905 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.273238
I20260806 07:58:29.854135 6755 tablet_replica.cc:333] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.854286 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 054c7be39877456fb2b196b5a75610bb
I20260806 07:58:29.854368 6755 raft_consensus.cc:2243] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.854585 6755 raft_consensus.cc:2272] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.855042 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 054c7be39877456fb2b196b5a75610bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.855631 6755 ts_tablet_manager.cc:1916] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.857043 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 054c7be39877456fb2b196b5a75610bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.857473 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 3036ed4cf6f04ec88c613f1bec3255d0
I20260806 07:58:29.857618 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":10188}
I20260806 07:58:29.857679 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":14344}
I20260806 07:58:29.857781 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 054c7be39877456fb2b196b5a75610bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.858629 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 3036ed4cf6f04ec88c613f1bec3255d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.859565 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 3036ed4cf6f04ec88c613f1bec3255d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.860225 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 3036ed4cf6f04ec88c613f1bec3255d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.860666 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet d2e9449a49ea445b89274f38ddca53ac
I20260806 07:58:29.861574 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet d2e9449a49ea445b89274f38ddca53ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.862457 6755 ts_tablet_manager.cc:1929] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.862607 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet d2e9449a49ea445b89274f38ddca53ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.862654 6755 log.cc:1199] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/a30189879fbf49c796eea2fe621b9d9a
I20260806 07:58:29.863122 6755 ts_tablet_manager.cc:1950] T a30189879fbf49c796eea2fe621b9d9a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:29.863270 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet d2e9449a49ea445b89274f38ddca53ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.863612 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 0f4bee6c27bf4e9198f13f81ea27678e
I20260806 07:58:29.864315 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f4bee6c27bf4e9198f13f81ea27678e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.864519 6648 ts_tablet_manager.cc:1929] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.864646 6648 log.cc:1199] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/a30189879fbf49c796eea2fe621b9d9a
I20260806 07:58:29.865031 6648 ts_tablet_manager.cc:1950] T a30189879fbf49c796eea2fe621b9d9a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.865340 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f4bee6c27bf4e9198f13f81ea27678e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.865989 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f4bee6c27bf4e9198f13f81ea27678e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.866418 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet e4de34f839a442818e8c8045000f7cca
I20260806 07:58:29.867421 6648 tablet_replica.cc:333] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.867532 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet a30189879fbf49c796eea2fe621b9d9a (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.867679 6648 raft_consensus.cc:2243] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 07:58:29.867808 6001 catalog_manager.cc:4995] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): delete failed for tablet a30189879fbf49c796eea2fe621b9d9a because the tablet was not found. No further retry: Not found: Tablet not found: a30189879fbf49c796eea2fe621b9d9a
I20260806 07:58:29.867873 6648 raft_consensus.cc:2272] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.868103 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet e4de34f839a442818e8c8045000f7cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.868402 6866 tablet_replica.cc:333] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.868702 6866 raft_consensus.cc:2243] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.868956 6648 ts_tablet_manager.cc:1916] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.869084 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet e4de34f839a442818e8c8045000f7cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.868963 6866 raft_consensus.cc:2272] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.869680 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet e4de34f839a442818e8c8045000f7cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.869850 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet d50d72195b4743d194125a7b231fcc80
I20260806 07:58:29.870611 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet d50d72195b4743d194125a7b231fcc80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.870621 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet d50d72195b4743d194125a7b231fcc80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.871340 6866 ts_tablet_manager.cc:1916] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.873857 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 4f3dc4525dae4b5f83c33b0be1ac01f6
I20260806 07:58:29.874667 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.874735 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet de9192eb384944ba9836187ba27edb38
I20260806 07:58:29.875248 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.875510 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet de9192eb384944ba9836187ba27edb38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.875535 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 33b87b2e51024e3d8311bbac25ae573b
I20260806 07:58:29.875459 6572 consensus_queue.cc:1048] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.875911 6648 ts_tablet_manager.cc:1929] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.876021 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.876451 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet f054c628591a4ef7ab8d607abd77a984
I20260806 07:58:29.876539 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet de9192eb384944ba9836187ba27edb38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.876360 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.003s sys 0.002s Metrics: {"bytes_written":609,"delete_count":0,"lbm_write_time_us":191,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":8}
I20260806 07:58:29.877135 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet f054c628591a4ef7ab8d607abd77a984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.877506 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet b5fe3aedfb004f9185b6307e0c087b63
I20260806 07:58:29.878000 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet f054c628591a4ef7ab8d607abd77a984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.878221 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 33b87b2e51024e3d8311bbac25ae573b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.878279 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet b5fe3aedfb004f9185b6307e0c087b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.878475 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 33b87b2e51024e3d8311bbac25ae573b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.879320 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet d50d72195b4743d194125a7b231fcc80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.881840 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 39fcd48f42fc4b0a9edc04adcc8364a0
I20260806 07:58:29.881958 6648 log.cc:1199] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/054c7be39877456fb2b196b5a75610bb
I20260806 07:58:29.882462 6648 ts_tablet_manager.cc:1950] T 054c7be39877456fb2b196b5a75610bb P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.882581 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet de9192eb384944ba9836187ba27edb38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.882784 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet fd1e9ae4d73d46d99f50b701d63d4f18
I20260806 07:58:29.883190 6152 tablet_service.cc:1558] Processing DeleteTablet for tablet 33b87b2e51024e3d8311bbac25ae573b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.883498 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet fd1e9ae4d73d46d99f50b701d63d4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.883620 6648 tablet_replica.cc:333] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.883742 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet b5fe3aedfb004f9185b6307e0c087b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.883823 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 43e15cc19276474a8b160d7959d8bf89
I20260806 07:58:29.883855 6648 raft_consensus.cc:2243] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.884147 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 39fcd48f42fc4b0a9edc04adcc8364a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.884207 6648 raft_consensus.cc:2272] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.883648 6160 tablet_service.cc:1558] Processing DeleteTablet for tablet f054c628591a4ef7ab8d607abd77a984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.884678 6014 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 7d68050d52894d868b86370cd5e449b8
I20260806 07:58:29.885217 6648 ts_tablet_manager.cc:1916] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.885473 6572 raft_consensus.cc:493] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.885519 6287 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d68050d52894d868b86370cd5e449b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.885700 6572 raft_consensus.cc:515] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.886319 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 39fcd48f42fc4b0a9edc04adcc8364a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.886514 6572 leader_election.cc:290] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.886848 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet fd1e9ae4d73d46d99f50b701d63d4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.887627 6285 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e15cc19276474a8b160d7959d8bf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:29.887802 6866 ts_tablet_manager.cc:1929] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.887919 6866 log.cc:1199] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/a30189879fbf49c796eea2fe621b9d9a
I20260806 07:58:29.888371 6866 ts_tablet_manager.cc:1950] T a30189879fbf49c796eea2fe621b9d9a P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.889088 6155 tablet_service.cc:1558] Processing DeleteTablet for tablet fd1e9ae4d73d46d99f50b701d63d4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.889273 6152 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e15cc19276474a8b160d7959d8bf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.889344 6866 tablet_replica.cc:333] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.889472 6147 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d68050d52894d868b86370cd5e449b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.889680 6160 tablet_service.cc:1558] Processing DeleteTablet for tablet 39fcd48f42fc4b0a9edc04adcc8364a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:29.889565 6866 raft_consensus.cc:2243] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.889919 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 054c7be39877456fb2b196b5a75610bb (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.890084 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.890225 6866 raft_consensus.cc:2272] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.890341 6453 raft_consensus.cc:2393] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:29.890730 6428 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e15cc19276474a8b160d7959d8bf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.890977 6430 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d68050d52894d868b86370cd5e449b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.891196 6866 ts_tablet_manager.cc:1916] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.891312 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet a30189879fbf49c796eea2fe621b9d9a (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.892110 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9192eb384944ba9836187ba27edb38" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.892344 6718 consensus_queue.cc:1048] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.892789 6718 consensus_queue.cc:1048] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.893117 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet b5fe3aedfb004f9185b6307e0c087b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:29 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:29.893353 6252 leader_election.cc:304] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.893641 6014 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46090:
name: "table-3"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260806 07:58:29.893878 6718 raft_consensus.cc:3060] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:58:29.894562 6014 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 07:58:29.897063 6718 raft_consensus.cc:2749] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.898262 6866 ts_tablet_manager.cc:1929] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.898516 6866 log.cc:1199] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/054c7be39877456fb2b196b5a75610bb
I20260806 07:58:29.898264 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.040s user 0.003s sys 0.011s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":19119,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":3991,"dirs.run_cpu_time_us":297,"dirs.run_wall_time_us":5199,"lbm_read_time_us":645,"lbm_reads_lt_1ms":29,"lbm_write_time_us":411,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":16560,"threads_started":1,"update_count":33}
I20260806 07:58:29.899168 6648 ts_tablet_manager.cc:1929] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.899550 6866 ts_tablet_manager.cc:1950] T 054c7be39877456fb2b196b5a75610bb P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.901953 6866 tablet_replica.cc:333] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.902232 6866 raft_consensus.cc:2243] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.902388 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 054c7be39877456fb2b196b5a75610bb (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.902554 6866 raft_consensus.cc:2272] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.902835 6592 consensus_queue.cc:1048] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.903936 6648 log.cc:1199] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/3036ed4cf6f04ec88c613f1bec3255d0
I20260806 07:58:29.904454 6648 ts_tablet_manager.cc:1950] T 3036ed4cf6f04ec88c613f1bec3255d0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.905304 6866 ts_tablet_manager.cc:1916] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.905328 6592 raft_consensus.cc:493] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:29.905469 6592 raft_consensus.cc:515] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:29.906215 6592 leader_election.cc:290] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:29.906637 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1e9ae4d73d46d99f50b701d63d4f18" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:29.906957 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1e9ae4d73d46d99f50b701d63d4f18" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:29.907245 6180 raft_consensus.cc:2393] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:29.907837 6252 leader_election.cc:304] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:29.907964 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.063s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":62840}
I20260806 07:58:29.908329 6648 tablet_replica.cc:333] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.908345 6592 raft_consensus.cc:3060] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:29.908303 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.063s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":62774}
I20260806 07:58:29.908496 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 3036ed4cf6f04ec88c613f1bec3255d0 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.908553 6648 raft_consensus.cc:2243] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.908737 6648 raft_consensus.cc:2272] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.909716 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.009s sys 0.003s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":19283,"cfile_init":6,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":26556,"dirs.run_cpu_time_us":319,"dirs.run_wall_time_us":1802,"lbm_read_time_us":618,"lbm_reads_lt_1ms":33,"lbm_write_time_us":346,"lbm_writes_lt_1ms":9,"mutex_wait_us":36494,"peak_mem_usage":32768,"reinsert_count":0,"update_count":20}
I20260806 07:58:29.911139 6592 raft_consensus.cc:2749] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:29.911808 6648 ts_tablet_manager.cc:1916] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.914106 6866 ts_tablet_manager.cc:1929] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.914330 6866 log.cc:1199] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/3036ed4cf6f04ec88c613f1bec3255d0
I20260806 07:58:29.915030 6866 ts_tablet_manager.cc:1950] T 3036ed4cf6f04ec88c613f1bec3255d0 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.916572 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 3036ed4cf6f04ec88c613f1bec3255d0 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.917161 6866 tablet_replica.cc:333] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.917488 6866 raft_consensus.cc:2243] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.917985 6866 raft_consensus.cc:2272] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.919268 6866 ts_tablet_manager.cc:1916] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.923003 6755 tablet_replica.cc:333] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:29.923291 6755 raft_consensus.cc:2243] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.923588 6755 raft_consensus.cc:2272] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.923857 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet a30189879fbf49c796eea2fe621b9d9a (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.926390 6866 ts_tablet_manager.cc:1929] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.926609 6866 log.cc:1199] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/d2e9449a49ea445b89274f38ddca53ac
I20260806 07:58:29.927136 6866 ts_tablet_manager.cc:1950] T d2e9449a49ea445b89274f38ddca53ac P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.928397 6866 tablet_replica.cc:333] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.928704 6866 raft_consensus.cc:2243] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:29.929203 6866 raft_consensus.cc:2272] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.931800 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.931813 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:29.932374 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet d2e9449a49ea445b89274f38ddca53ac (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.934836 6866 ts_tablet_manager.cc:1916] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.853808 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.001s sys 0.003s Metrics: {"bytes_written":547,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":5}
I20260806 07:58:29.940106 6648 ts_tablet_manager.cc:1929] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.940285 6648 log.cc:1199] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/d2e9449a49ea445b89274f38ddca53ac
I20260806 07:58:29.940776 6648 ts_tablet_manager.cc:1950] T d2e9449a49ea445b89274f38ddca53ac P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:29.945022 6866 ts_tablet_manager.cc:1929] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.945308 6866 log.cc:1199] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/0f4bee6c27bf4e9198f13f81ea27678e
I20260806 07:58:29.946019 6866 ts_tablet_manager.cc:1950] T 0f4bee6c27bf4e9198f13f81ea27678e P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.947114 6623 consensus_queue.cc:1048] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.947623 6866 tablet_replica.cc:333] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:29.948046 6866 raft_consensus.cc:2243] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.948452 6866 raft_consensus.cc:2272] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.949625 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet d2e9449a49ea445b89274f38ddca53ac (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.947278 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 0f4bee6c27bf4e9198f13f81ea27678e (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:29.951709 6866 ts_tablet_manager.cc:1916] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.952342 6648 tablet_replica.cc:333] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:29.952641 6648 raft_consensus.cc:2243] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:29.952849 6648 raft_consensus.cc:2272] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:29.954209 6648 ts_tablet_manager.cc:1916] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.962347 6755 ts_tablet_manager.cc:1916] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:29.962577 6568 consensus_queue.cc:1048] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.973660 6866 ts_tablet_manager.cc:1929] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.974063 6866 log.cc:1199] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/e4de34f839a442818e8c8045000f7cca
I20260806 07:58:29.975721 6866 ts_tablet_manager.cc:1950] T e4de34f839a442818e8c8045000f7cca P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:29.979135 6648 ts_tablet_manager.cc:1929] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:29.979302 6623 consensus_queue.cc:1048] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.979421 6648 log.cc:1199] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/0f4bee6c27bf4e9198f13f81ea27678e
I20260806 07:58:29.983385 6422 tablet_service.cc:1511] Processing CreateTablet for tablet 5e6d749166f540a0a83818c2ef5a00e5 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:29.983876 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6d749166f540a0a83818c2ef5a00e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:29.983913 6285 tablet_service.cc:1511] Processing CreateTablet for tablet 5e6d749166f540a0a83818c2ef5a00e5 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:29.984457 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6d749166f540a0a83818c2ef5a00e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:29.985322 6287 tablet_service.cc:1511] Processing CreateTablet for tablet 5eef6e56fda745a3aa131e3db6a0d13d (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:29.985760 6287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eef6e56fda745a3aa131e3db6a0d13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:29.990633 6160 tablet_service.cc:1511] Processing CreateTablet for tablet 5e6d749166f540a0a83818c2ef5a00e5 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:29.990777 6623 consensus_queue.cc:1048] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:29.991082 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6d749166f540a0a83818c2ef5a00e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:29.993944 6147 tablet_service.cc:1511] Processing CreateTablet for tablet 5eef6e56fda745a3aa131e3db6a0d13d (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:29.994333 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eef6e56fda745a3aa131e3db6a0d13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:29.994474 6531 tablet_bootstrap.cc:492] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:29.996930 6531 tablet_bootstrap.cc:654] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:29.999693 6430 tablet_service.cc:1511] Processing CreateTablet for tablet 5eef6e56fda745a3aa131e3db6a0d13d (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.000159 6430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eef6e56fda745a3aa131e3db6a0d13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.000739 6152 tablet_service.cc:1511] Processing CreateTablet for tablet d1a3a6825393442a8fa5c96a230afa26 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.001142 6152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1a3a6825393442a8fa5c96a230afa26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.005574 6427 tablet_service.cc:1511] Processing CreateTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.005954 6428 tablet_service.cc:1511] Processing CreateTablet for tablet d1a3a6825393442a8fa5c96a230afa26 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.006155 6427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c454c14d784d5f9acb483fc8d26ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.006409 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1a3a6825393442a8fa5c96a230afa26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.006383 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.069s user 0.015s sys 0.001s Metrics: {"bytes_written":718,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":49,"dirs.run_cpu_time_us":178,"dirs.run_wall_time_us":1280,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1748,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":17,"update_count":16}
I20260806 07:58:30.009941 6296 tablet_service.cc:1511] Processing CreateTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.008852 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.077s user 0.016s sys 0.000s Metrics: {"bytes_written":718,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":4540,"dirs.run_cpu_time_us":171,"dirs.run_wall_time_us":1086,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":33476,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":17,"update_count":16}
I20260806 07:58:30.011057 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:30.011129 6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c454c14d784d5f9acb483fc8d26ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.011409 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.360261
I20260806 07:58:30.011708 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.360261
I20260806 07:58:30.011926 6284 tablet_service.cc:1511] Processing CreateTablet for tablet 81e7cd5a08d94c519285f18f66c01f0d (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.012042 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:30.012389 6284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e7cd5a08d94c519285f18f66c01f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.013070 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":126,"lbm_reads_lt_1ms":4}
I20260806 07:58:30.016083 6421 tablet_service.cc:1511] Processing CreateTablet for tablet d2582adff5654c49b98f072ddfa3ff49 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.016268 6295 tablet_service.cc:1511] Processing CreateTablet for tablet d1a3a6825393442a8fa5c96a230afa26 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.016678 6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1a3a6825393442a8fa5c96a230afa26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.016750 6421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2582adff5654c49b98f072ddfa3ff49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.017912 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.018252 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.018590 6425 tablet_service.cc:1511] Processing CreateTablet for tablet 7f80b3d857bd4e62b71bdd63157e8872 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.018729 6293 tablet_service.cc:1511] Processing CreateTablet for tablet 7f80b3d857bd4e62b71bdd63157e8872 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.019146 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f80b3d857bd4e62b71bdd63157e8872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.019385 6294 tablet_service.cc:1511] Processing CreateTablet for tablet 2746179ffc5641c393439e5292ac0b69 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.020237 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2746179ffc5641c393439e5292ac0b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.020972 6288 tablet_service.cc:1511] Processing CreateTablet for tablet d2582adff5654c49b98f072ddfa3ff49 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.021402 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2582adff5654c49b98f072ddfa3ff49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.008401 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:30.022385 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:30.022684 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:30.022866 6755 ts_tablet_manager.cc:1929] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.022995 6755 log.cc:1199] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/054c7be39877456fb2b196b5a75610bb
I20260806 07:58:30.023098 6425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f80b3d857bd4e62b71bdd63157e8872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.023312 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20260806 07:58:30.023742 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.023990 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.025300 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.293704
I20260806 07:58:30.025908 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.293704
I20260806 07:58:30.026803 6429 tablet_service.cc:1511] Processing CreateTablet for tablet b13ae96de4eb42088b187c4efc95802a (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.028231 6531 tablet_bootstrap.cc:492] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.029801 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet e4de34f839a442818e8c8045000f7cca (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.030611 6433 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6182cc1daf4f6ca2c2464b6977cde1 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.031327 6423 tablet_service.cc:1511] Processing CreateTablet for tablet 2746179ffc5641c393439e5292ac0b69 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.031769 6531 ts_tablet_manager.cc:1403] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.005s
I20260806 07:58:30.032688 6424 tablet_service.cc:1511] Processing CreateTablet for tablet be96d575a672438994b6c6364180ec84 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.035147 6432 tablet_service.cc:1511] Processing CreateTablet for tablet dbc52165692848699c405d3e54739402 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.031237 6422 tablet_service.cc:1511] Processing CreateTablet for tablet 81e7cd5a08d94c519285f18f66c01f0d (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.038249 6648 ts_tablet_manager.cc:1950] T 0f4bee6c27bf4e9198f13f81ea27678e P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.041812 6149 tablet_service.cc:1511] Processing CreateTablet for tablet 7f80b3d857bd4e62b71bdd63157e8872 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.042312 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f80b3d857bd4e62b71bdd63157e8872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.042269 6154 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6182cc1daf4f6ca2c2464b6977cde1 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.042693 6154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6182cc1daf4f6ca2c2464b6977cde1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.042927 6148 tablet_service.cc:1511] Processing CreateTablet for tablet d2582adff5654c49b98f072ddfa3ff49 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.043960 6282 tablet_service.cc:1511] Processing CreateTablet for tablet b13ae96de4eb42088b187c4efc95802a (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.045085 6755 ts_tablet_manager.cc:1950] T 054c7be39877456fb2b196b5a75610bb P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.045297 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2582adff5654c49b98f072ddfa3ff49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.046433 6286 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6182cc1daf4f6ca2c2464b6977cde1 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.047130 6157 tablet_service.cc:1511] Processing CreateTablet for tablet be96d575a672438994b6c6364180ec84 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.047466 6156 tablet_service.cc:1511] Processing CreateTablet for tablet 2746179ffc5641c393439e5292ac0b69 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.047689 6153 tablet_service.cc:1511] Processing CreateTablet for tablet b13ae96de4eb42088b187c4efc95802a (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.049100 6155 tablet_service.cc:1511] Processing CreateTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
W20260806 07:58:30.050693 6114 consensus_peers.cc:597] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e4de34f839a442818e8c8045000f7cca. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:30.051393 6531 raft_consensus.cc:359] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.052446 6282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13ae96de4eb42088b187c4efc95802a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.053778 6150 tablet_service.cc:1511] Processing CreateTablet for tablet c31bd22c7f6e47c595747dec77a8cc70 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.054471 6157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be96d575a672438994b6c6364180ec84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.054967 6866 tablet_replica.cc:333] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.055194 6866 raft_consensus.cc:2243] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.055298 6283 tablet_service.cc:1511] Processing CreateTablet for tablet be96d575a672438994b6c6364180ec84 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.055714 6426 tablet_service.cc:1511] Processing CreateTablet for tablet c31bd22c7f6e47c595747dec77a8cc70 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.056277 6420 tablet_service.cc:1511] Processing CreateTablet for tablet f9aa031a25104549905f7a7e1007187f (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.056718 6419 tablet_service.cc:1511] Processing CreateTablet for tablet a1ce9afe0d394368a97a22242d3e403f (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.057199 6531 raft_consensus.cc:385] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.057197 6418 tablet_service.cc:1511] Processing CreateTablet for tablet 85cd55230a604f4c8c22594e2df60249 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.057344 6531 raft_consensus.cc:740] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.057688 6531 consensus_queue.cc:260] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.055667 6429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13ae96de4eb42088b187c4efc95802a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.058370 6531 ts_tablet_manager.cc:1434] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260806 07:58:30.058939 6531 tablet_bootstrap.cc:492] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.060820 6531 tablet_bootstrap.cc:654] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.062431 6433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6182cc1daf4f6ca2c2464b6977cde1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.063123 6866 raft_consensus.cc:2272] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.064106 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2746179ffc5641c393439e5292ac0b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.065796 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc52165692848699c405d3e54739402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.070093 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be96d575a672438994b6c6364180ec84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.073046 6151 tablet_service.cc:1511] Processing CreateTablet for tablet 85cd55230a604f4c8c22594e2df60249 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.073879 6158 tablet_service.cc:1511] Processing CreateTablet for tablet dbc52165692848699c405d3e54739402 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.075634 6418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85cd55230a604f4c8c22594e2df60249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.077036 5929 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:30.077713 6420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9aa031a25104549905f7a7e1007187f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.081907 6419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1ce9afe0d394368a97a22242d3e403f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.086699 6426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31bd22c7f6e47c595747dec77a8cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.093228 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e7cd5a08d94c519285f18f66c01f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.105238 6866 ts_tablet_manager.cc:1916] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.116609 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 0f4bee6c27bf4e9198f13f81ea27678e (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.126385 6648 tablet_replica.cc:333] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.126350 6544 consensus_queue.cc:1048] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260806 07:58:30.126622 6145 tablet_service.cc:1511] Processing CreateTablet for tablet a1ce9afe0d394368a97a22242d3e403f (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.128106 6779 consensus_queue.cc:1048] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260806 07:58:30.129513 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.100s user 0.010s sys 0.006s Metrics: {"dirs.queue_time_us":1779,"dirs.run_cpu_time_us":830,"dirs.run_wall_time_us":4364,"lbm_write_time_us":435,"lbm_writes_lt_1ms":7,"num_input_rowsets":0,"thread_start_us":41862,"threads_started":1}
I20260806 07:58:30.131067 6159 tablet_service.cc:1511] Processing CreateTablet for tablet f9aa031a25104549905f7a7e1007187f (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.131104 6146 tablet_service.cc:1511] Processing CreateTablet for tablet 81e7cd5a08d94c519285f18f66c01f0d (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.132143 6156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2746179ffc5641c393439e5292ac0b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.135768 6150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31bd22c7f6e47c595747dec77a8cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.136721 6648 raft_consensus.cc:2243] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.140269 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e7cd5a08d94c519285f18f66c01f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.145987 6648 raft_consensus.cc:2272] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.147666 6648 ts_tablet_manager.cc:1916] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.156810 6286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6182cc1daf4f6ca2c2464b6977cde1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.157929 6283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be96d575a672438994b6c6364180ec84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.158583 6592 raft_consensus.cc:493] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.158735 6592 raft_consensus.cc:515] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.159443 6291 tablet_service.cc:1511] Processing CreateTablet for tablet c31bd22c7f6e47c595747dec77a8cc70 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.159864 6291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c31bd22c7f6e47c595747dec77a8cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.160099 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d68050d52894d868b86370cd5e449b8" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.160778 6159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9aa031a25104549905f7a7e1007187f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.161923 6151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85cd55230a604f4c8c22594e2df60249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.162140 6145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1ce9afe0d394368a97a22242d3e403f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.163343 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b13ae96de4eb42088b187c4efc95802a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.163782 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c454c14d784d5f9acb483fc8d26ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.167604 6158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc52165692848699c405d3e54739402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.170017 6755 tablet_replica.cc:333] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.173435 6281 tablet_service.cc:1511] Processing CreateTablet for tablet dbc52165692848699c405d3e54739402 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.174431 6755 raft_consensus.cc:2243] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.174657 6755 raft_consensus.cc:2272] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.176079 6531 tablet_bootstrap.cc:492] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.176247 6531 ts_tablet_manager.cc:1403] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.118s user 0.017s sys 0.019s
I20260806 07:58:30.177186 6531 raft_consensus.cc:359] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.177393 6531 raft_consensus.cc:385] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.177492 6531 raft_consensus.cc:740] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.177783 6531 consensus_queue.cc:260] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.178323 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4334 bytes on disk
I20260806 07:58:30.178359 6531 ts_tablet_manager.cc:1434] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:30.178622 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4334 bytes on disk
I20260806 07:58:30.179322 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4}
I20260806 07:58:30.179723 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.181190 6623 consensus_queue.cc:1048] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:30.182122 6532 tablet_bootstrap.cc:492] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.184069 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 054c7be39877456fb2b196b5a75610bb (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.131569 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.118s user 0.014s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":1698,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":89,"dirs.run_cpu_time_us":128,"dirs.run_wall_time_us":569,"lbm_read_time_us":388,"lbm_reads_lt_1ms":21,"lbm_write_time_us":369,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":16}
I20260806 07:58:30.185998 6287 tablet_service.cc:1511] Processing CreateTablet for tablet a1ce9afe0d394368a97a22242d3e403f (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.187330 6281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc52165692848699c405d3e54739402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.191401 6531 tablet_bootstrap.cc:492] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.193248 6287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1ce9afe0d394368a97a22242d3e403f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.199004 6289 tablet_service.cc:1511] Processing CreateTablet for tablet f9aa031a25104549905f7a7e1007187f (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.199546 6290 tablet_service.cc:1511] Processing CreateTablet for tablet 85cd55230a604f4c8c22594e2df60249 (DEFAULT_TABLE table=table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:30.199954 6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85cd55230a604f4c8c22594e2df60249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.201985 6314 raft_consensus.cc:3060] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.203447 6531 tablet_bootstrap.cc:654] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.205029 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d68050d52894d868b86370cd5e449b8" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.205119 6755 ts_tablet_manager.cc:1916] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.205476 6592 leader_election.cc:290] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.206100 6866 ts_tablet_manager.cc:1929] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.206239 6866 log.cc:1199] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/4f3dc4525dae4b5f83c33b0be1ac01f6
I20260806 07:58:30.206934 6866 ts_tablet_manager.cc:1950] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.208027 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.208376 6866 tablet_replica.cc:333] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.208614 6866 raft_consensus.cc:2243] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.208748 6289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9aa031a25104549905f7a7e1007187f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:30.208787 6866 pending_rounds.cc:70] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Trying to abort 1 pending ops.
I20260806 07:58:30.208864 6866 pending_rounds.cc:77] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468737102884864 op_type: NO_OP noop_request { }
I20260806 07:58:30.209962 6250 leader_election.cc:304] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:30.215237 6866 raft_consensus.cc:2889] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:30.215392 6866 raft_consensus.cc:2272] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.215663 6779 consensus_queue.cc:1048] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260806 07:58:30.218319 6866 ts_tablet_manager.cc:1916] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.218434 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.192s user 0.030s sys 0.011s Metrics: {"bytes_written":32226,"cfile_cache_hit":14,"cfile_cache_hit_bytes":21610,"cfile_cache_miss":34,"cfile_cache_miss_bytes":31260,"cfile_init":9,"delta_iterators_relevant":8,"dirs.queue_time_us":15254,"dirs.run_cpu_time_us":643,"dirs.run_wall_time_us":5203,"drs_written":1,"lbm_read_time_us":1764,"lbm_reads_lt_1ms":70,"lbm_write_time_us":1157,"lbm_writes_lt_1ms":28,"mutex_wait_us":8793,"num_input_rowsets":2,"peak_mem_usage":140265,"rows_written":71,"thread_start_us":2934,"threads_started":1}
I20260806 07:58:30.225790 6866 ts_tablet_manager.cc:1929] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.225920 6866 log.cc:1199] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/de9192eb384944ba9836187ba27edb38
I20260806 07:58:30.226418 6866 ts_tablet_manager.cc:1950] T de9192eb384944ba9836187ba27edb38 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.227393 6866 tablet_replica.cc:333] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.227592 6866 raft_consensus.cc:2243] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.227890 6866 raft_consensus.cc:2272] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.228271 6544 consensus_queue.cc:1048] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260806 07:58:30.228420 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet de9192eb384944ba9836187ba27edb38 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.229487 6879 tablet_bootstrap.cc:492] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.239605 6531 tablet_bootstrap.cc:492] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.239836 6531 ts_tablet_manager.cc:1403] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.000s
I20260806 07:58:30.239571 6866 ts_tablet_manager.cc:1916] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.242005 6531 raft_consensus.cc:359] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.244129 6531 raft_consensus.cc:385] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.244280 6531 raft_consensus.cc:740] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.244810 6531 consensus_queue.cc:260] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.246671 6531 ts_tablet_manager.cc:1434] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 07:58:30.247040 6531 tablet_bootstrap.cc:492] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.252434 6531 tablet_bootstrap.cc:654] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.254287 6544 consensus_queue.cc:1048] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:30.255380 6532 tablet_bootstrap.cc:654] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.261327 6866 ts_tablet_manager.cc:1929] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.261837 6866 log.cc:1199] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/f054c628591a4ef7ab8d607abd77a984
I20260806 07:58:30.262547 6866 ts_tablet_manager.cc:1950] T f054c628591a4ef7ab8d607abd77a984 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.263696 6866 tablet_replica.cc:333] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.263906 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet f054c628591a4ef7ab8d607abd77a984 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.263985 6592 raft_consensus.cc:2749] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.266273 6866 raft_consensus.cc:2243] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.266505 6866 raft_consensus.cc:2272] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.267145 6866 ts_tablet_manager.cc:1916] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.268613 6531 tablet_bootstrap.cc:492] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.268826 6531 ts_tablet_manager.cc:1403] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.002s
I20260806 07:58:30.269753 6531 raft_consensus.cc:359] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.270018 6531 raft_consensus.cc:385] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.270205 6531 raft_consensus.cc:740] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.270864 6531 consensus_queue.cc:260] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.272848 6531 ts_tablet_manager.cc:1434] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:30.273352 6531 tablet_bootstrap.cc:492] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.275619 6648 ts_tablet_manager.cc:1929] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.281702 6531 tablet_bootstrap.cc:654] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.285796 6648 log.cc:1199] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/e4de34f839a442818e8c8045000f7cca
I20260806 07:58:30.286305 6648 ts_tablet_manager.cc:1950] T e4de34f839a442818e8c8045000f7cca P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.287494 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet e4de34f839a442818e8c8045000f7cca (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.282334 6879 tablet_bootstrap.cc:654] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.293993 6648 tablet_replica.cc:333] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.294270 6648 raft_consensus.cc:2243] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.294534 6648 raft_consensus.cc:2272] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.297142 6648 ts_tablet_manager.cc:1916] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.303089 6568 consensus_queue.cc:1048] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 07:58:30.305784 6114 consensus_peers.cc:597] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de9192eb384944ba9836187ba27edb38. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:30.306203 6866 ts_tablet_manager.cc:1929] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 07:58:30.306339 6866 log.cc:1199] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/33b87b2e51024e3d8311bbac25ae573b
I20260806 07:58:30.306978 6866 ts_tablet_manager.cc:1950] T 33b87b2e51024e3d8311bbac25ae573b P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
W20260806 07:58:30.307377 6114 consensus_peers.cc:597] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:30.310177 6866 tablet_replica.cc:333] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.310390 6866 raft_consensus.cc:2243] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.310590 6866 raft_consensus.cc:2272] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.311630 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 33b87b2e51024e3d8311bbac25ae573b (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.311859 6866 ts_tablet_manager.cc:1916] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.313073 6879 tablet_bootstrap.cc:492] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.313217 6879 ts_tablet_manager.cc:1403] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.085s user 0.008s sys 0.003s
I20260806 07:58:30.314064 6879 raft_consensus.cc:359] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.314272 6879 raft_consensus.cc:385] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.314365 6879 raft_consensus.cc:740] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.314656 6879 consensus_queue.cc:260] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.317265 6879 ts_tablet_manager.cc:1434] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:30.319499 6879 tablet_bootstrap.cc:492] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.320945 6866 ts_tablet_manager.cc:1929] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.321106 6866 log.cc:1199] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/d50d72195b4743d194125a7b231fcc80
I20260806 07:58:30.321426 6879 tablet_bootstrap.cc:654] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.321403 6755 ts_tablet_manager.cc:1929] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.321508 6866 ts_tablet_manager.cc:1950] T d50d72195b4743d194125a7b231fcc80 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.322499 6866 tablet_replica.cc:333] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.322696 6866 raft_consensus.cc:2243] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.322991 6866 raft_consensus.cc:2272] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.323196 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet d50d72195b4743d194125a7b231fcc80 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.326190 6866 ts_tablet_manager.cc:1916] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.328325 6648 ts_tablet_manager.cc:1929] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.328452 6648 log.cc:1199] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/d50d72195b4743d194125a7b231fcc80
I20260806 07:58:30.328908 6648 ts_tablet_manager.cc:1950] T d50d72195b4743d194125a7b231fcc80 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.334363 6583 consensus_queue.cc:1048] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:30.335850 6866 ts_tablet_manager.cc:1929] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.336057 6866 log.cc:1199] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/39fcd48f42fc4b0a9edc04adcc8364a0
I20260806 07:58:30.336673 6866 ts_tablet_manager.cc:1950] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.338032 6866 tablet_replica.cc:333] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.338335 6866 raft_consensus.cc:2243] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.338783 6866 raft_consensus.cc:2272] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.339239 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 39fcd48f42fc4b0a9edc04adcc8364a0 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.346611 6755 log.cc:1199] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/3036ed4cf6f04ec88c613f1bec3255d0
I20260806 07:58:30.347343 6755 ts_tablet_manager.cc:1950] T 3036ed4cf6f04ec88c613f1bec3255d0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.348584 6755 tablet_replica.cc:333] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.348863 6755 raft_consensus.cc:2243] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.349102 6755 raft_consensus.cc:2272] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.346413 6532 tablet_bootstrap.cc:492] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.349471 6532 ts_tablet_manager.cc:1403] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.180s user 0.007s sys 0.005s
I20260806 07:58:30.350206 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 3036ed4cf6f04ec88c613f1bec3255d0 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.350643 6532 raft_consensus.cc:359] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.350926 6532 raft_consensus.cc:385] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.351027 6532 raft_consensus.cc:740] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.351301 6755 ts_tablet_manager.cc:1916] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.351419 6532 consensus_queue.cc:260] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.352260 6532 ts_tablet_manager.cc:1434] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:30.352681 6532 tablet_bootstrap.cc:492] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.353960 6866 ts_tablet_manager.cc:1916] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.354746 6532 tablet_bootstrap.cc:654] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.365226 6885 consensus_queue.cc:1048] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:30.365621 6755 ts_tablet_manager.cc:1929] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.365773 6755 log.cc:1199] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/d2e9449a49ea445b89274f38ddca53ac
I20260806 07:58:30.365792 6531 tablet_bootstrap.cc:492] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.365819 6866 ts_tablet_manager.cc:1929] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.366024 6866 log.cc:1199] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/fd1e9ae4d73d46d99f50b701d63d4f18
I20260806 07:58:30.366071 6531 ts_tablet_manager.cc:1403] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.030s
I20260806 07:58:30.366259 6755 ts_tablet_manager.cc:1950] T d2e9449a49ea445b89274f38ddca53ac P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.366560 6866 ts_tablet_manager.cc:1950] T fd1e9ae4d73d46d99f50b701d63d4f18 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.367059 6531 raft_consensus.cc:359] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.367267 6531 raft_consensus.cc:385] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.367352 6531 raft_consensus.cc:740] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.367507 6866 tablet_replica.cc:333] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.367650 6531 consensus_queue.cc:260] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.367702 6866 raft_consensus.cc:2243] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.367892 6755 tablet_replica.cc:333] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.368088 6623 raft_consensus.cc:493] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.368268 6866 raft_consensus.cc:2272] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.368235 6755 raft_consensus.cc:2243] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.368224 6623 raft_consensus.cc:515] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.368530 6755 raft_consensus.cc:2272] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.368979 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet fd1e9ae4d73d46d99f50b701d63d4f18 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.369043 6623 leader_election.cc:290] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.371181 6866 ts_tablet_manager.cc:1916] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.372239 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2582adff5654c49b98f072ddfa3ff49" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.372408 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2582adff5654c49b98f072ddfa3ff49" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:30.372895 6384 leader_election.cc:343] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
W20260806 07:58:30.373078 6387 leader_election.cc:343] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.373176 6387 leader_election.cc:304] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:30.373628 6531 ts_tablet_manager.cc:1434] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 07:58:30.373628 6623 raft_consensus.cc:2749] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.374028 6531 tablet_bootstrap.cc:492] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.367338 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet d2e9449a49ea445b89274f38ddca53ac (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.375882 6755 ts_tablet_manager.cc:1916] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.379112 6531 tablet_bootstrap.cc:654] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.381132 6623 raft_consensus.cc:493] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.381291 6623 raft_consensus.cc:515] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.381994 6623 leader_election.cc:290] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.382959 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:30.383597 6384 leader_election.cc:343] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.384397 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:30.385047 6387 leader_election.cc:343] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.385195 6387 leader_election.cc:304] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:30.385767 6623 raft_consensus.cc:2749] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.386340 6879 tablet_bootstrap.cc:492] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.388376 6866 ts_tablet_manager.cc:1929] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.388566 6866 log.cc:1199] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/43e15cc19276474a8b160d7959d8bf89
I20260806 07:58:30.389164 6866 ts_tablet_manager.cc:1950] T 43e15cc19276474a8b160d7959d8bf89 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.389361 6879 ts_tablet_manager.cc:1403] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.002s
I20260806 07:58:30.390301 6879 raft_consensus.cc:359] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.390522 6879 raft_consensus.cc:385] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.390607 6879 raft_consensus.cc:740] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.390908 6866 tablet_replica.cc:333] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.391161 6866 raft_consensus.cc:2243] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.391369 6866 raft_consensus.cc:2272] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.392292 6866 ts_tablet_manager.cc:1916] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.390872 6879 consensus_queue.cc:260] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.394160 6879 ts_tablet_manager.cc:1434] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 07:58:30.394491 6879 tablet_bootstrap.cc:492] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.396283 6879 tablet_bootstrap.cc:654] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.398366 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 43e15cc19276474a8b160d7959d8bf89 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.399201 6755 ts_tablet_manager.cc:1929] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.399365 6755 log.cc:1199] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/0f4bee6c27bf4e9198f13f81ea27678e
I20260806 07:58:30.399806 6866 ts_tablet_manager.cc:1929] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.399876 6755 ts_tablet_manager.cc:1950] T 0f4bee6c27bf4e9198f13f81ea27678e P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.400063 6866 log.cc:1199] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/7d68050d52894d868b86370cd5e449b8
I20260806 07:58:30.400563 6866 ts_tablet_manager.cc:1950] T 7d68050d52894d868b86370cd5e449b8 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.401026 6755 tablet_replica.cc:333] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.401247 6755 raft_consensus.cc:2243] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.401458 6755 raft_consensus.cc:2272] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.401645 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 0f4bee6c27bf4e9198f13f81ea27678e (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.402009 6866 tablet_replica.cc:333] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:30.402197 6866 raft_consensus.cc:2243] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.402379 6866 raft_consensus.cc:2272] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.402456 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 7d68050d52894d868b86370cd5e449b8 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.402850 6531 tablet_bootstrap.cc:492] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.403002 6531 ts_tablet_manager.cc:1403] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.005s
I20260806 07:58:30.403772 6879 tablet_bootstrap.cc:492] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.403906 6879 ts_tablet_manager.cc:1403] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260806 07:58:30.404202 6531 raft_consensus.cc:359] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.404386 6531 raft_consensus.cc:385] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.404476 6531 raft_consensus.cc:740] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.404670 6879 raft_consensus.cc:359] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.404847 6879 raft_consensus.cc:385] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.404778 6531 consensus_queue.cc:260] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.404927 6879 raft_consensus.cc:740] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.405169 6879 consensus_queue.cc:260] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.405746 6879 ts_tablet_manager.cc:1434] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:30.406093 6531 ts_tablet_manager.cc:1434] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 07:58:30.406112 6879 tablet_bootstrap.cc:492] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.406360 6531 tablet_bootstrap.cc:492] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.408373 6866 ts_tablet_manager.cc:1916] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.409473 6879 tablet_bootstrap.cc:654] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.410167 6755 ts_tablet_manager.cc:1916] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.418596 6755 ts_tablet_manager.cc:1929] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.418797 6755 log.cc:1199] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/e4de34f839a442818e8c8045000f7cca
I20260806 07:58:30.419241 6755 ts_tablet_manager.cc:1950] T e4de34f839a442818e8c8045000f7cca P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.420374 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet e4de34f839a442818e8c8045000f7cca (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.420784 6755 tablet_replica.cc:333] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.421309 6755 raft_consensus.cc:2243] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.421761 6755 raft_consensus.cc:2272] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.423159 6532 tablet_bootstrap.cc:492] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.423300 6532 ts_tablet_manager.cc:1403] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.000s
I20260806 07:58:30.424042 6532 raft_consensus.cc:359] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.424219 6532 raft_consensus.cc:385] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.424294 6532 raft_consensus.cc:740] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.424489 6532 consensus_queue.cc:260] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.424948 6532 ts_tablet_manager.cc:1434] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 07:58:30.425238 6532 tablet_bootstrap.cc:492] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.428237 6866 ts_tablet_manager.cc:1929] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.428401 6866 log.cc:1199] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/b5fe3aedfb004f9185b6307e0c087b63
I20260806 07:58:30.428931 6866 ts_tablet_manager.cc:1950] T b5fe3aedfb004f9185b6307e0c087b63 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:30.429956 6755 ts_tablet_manager.cc:1916] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.430063 6531 tablet_bootstrap.cc:654] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.430090 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet b5fe3aedfb004f9185b6307e0c087b63 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.435978 6879 tablet_bootstrap.cc:492] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.436187 6879 ts_tablet_manager.cc:1403] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.006s
W20260806 07:58:30.437084 6250 consensus_peers.cc:597] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f3dc4525dae4b5f83c33b0be1ac01f6. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:30.437067 6879 raft_consensus.cc:359] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.437278 6879 raft_consensus.cc:385] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.437351 6879 raft_consensus.cc:740] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.437629 6879 consensus_queue.cc:260] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.438199 6879 ts_tablet_manager.cc:1434] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 07:58:30.438576 6879 tablet_bootstrap.cc:492] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.440410 6879 tablet_bootstrap.cc:654] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.438530 6532 tablet_bootstrap.cc:654] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.441128 6755 ts_tablet_manager.cc:1929] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.441540 6755 log.cc:1199] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/d50d72195b4743d194125a7b231fcc80
I20260806 07:58:30.442085 6755 ts_tablet_manager.cc:1950] T d50d72195b4743d194125a7b231fcc80 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.443257 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet d50d72195b4743d194125a7b231fcc80 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.443687 6755 tablet_replica.cc:333] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.443944 6755 raft_consensus.cc:2243] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.444175 6755 raft_consensus.cc:2272] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.445242 6755 ts_tablet_manager.cc:1916] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.449725 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet d50d72195b4743d194125a7b231fcc80 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.450402 6648 tablet_replica.cc:333] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.450888 6648 raft_consensus.cc:2243] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.451336 6648 raft_consensus.cc:2272] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.452546 6648 ts_tablet_manager.cc:1916] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.453464 6755 ts_tablet_manager.cc:1929] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.453692 6755 log.cc:1199] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/de9192eb384944ba9836187ba27edb38
I20260806 07:58:30.454635 6755 ts_tablet_manager.cc:1950] T de9192eb384944ba9836187ba27edb38 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.454869 6879 tablet_bootstrap.cc:492] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.455046 6879 ts_tablet_manager.cc:1403] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20260806 07:58:30.456082 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet de9192eb384944ba9836187ba27edb38 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.456080 6879 raft_consensus.cc:359] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.456297 6879 raft_consensus.cc:385] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.456429 6879 raft_consensus.cc:740] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.456648 6755 tablet_replica.cc:333] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.456701 6879 consensus_queue.cc:260] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.457255 6879 ts_tablet_manager.cc:1434] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:30.458096 6879 tablet_bootstrap.cc:492] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.458459 6755 raft_consensus.cc:2243] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.458911 6755 raft_consensus.cc:2272] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.460841 6544 raft_consensus.cc:493] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.461004 6544 raft_consensus.cc:515] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.461104 6755 ts_tablet_manager.cc:1916] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.461644 6544 leader_election.cc:290] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.461891 6879 tablet_bootstrap.cc:654] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.462555 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.463045 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.463267 6450 raft_consensus.cc:2468] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
W20260806 07:58:30.463783 6113 leader_election.cc:343] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.465850 6114 leader_election.cc:304] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.466642 6544 raft_consensus.cc:2804] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.466789 6544 raft_consensus.cc:493] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.466917 6544 raft_consensus.cc:3060] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.469853 6755 ts_tablet_manager.cc:1929] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.470099 6755 log.cc:1199] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/4f3dc4525dae4b5f83c33b0be1ac01f6
I20260806 07:58:30.470564 6544 raft_consensus.cc:515] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.470693 6755 ts_tablet_manager.cc:1950] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.471467 6532 tablet_bootstrap.cc:492] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.471807 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:30.472503 6113 leader_election.cc:343] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.472898 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.473135 6450 raft_consensus.cc:3060] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.473712 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.473922 6532 ts_tablet_manager.cc:1403] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.003s
I20260806 07:58:30.474781 6755 tablet_replica.cc:333] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.475026 6755 raft_consensus.cc:2243] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.475302 6755 pending_rounds.cc:70] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Trying to abort 1 pending ops.
I20260806 07:58:30.475422 6755 pending_rounds.cc:77] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315468736026619904 op_type: NO_OP noop_request { }
I20260806 07:58:30.475457 6532 raft_consensus.cc:359] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.475788 6755 raft_consensus.cc:2889] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 07:58:30.475939 6755 raft_consensus.cc:2272] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.476014 6532 raft_consensus.cc:385] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.476117 6532 raft_consensus.cc:740] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.476352 6450 raft_consensus.cc:2468] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.476346 6532 consensus_queue.cc:260] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.476866 6532 ts_tablet_manager.cc:1434] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:30.477192 6532 tablet_bootstrap.cc:492] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.477154 6114 leader_election.cc:304] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.477483 6544 leader_election.cc:290] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.477550 6779 raft_consensus.cc:2804] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.477881 6779 raft_consensus.cc:697] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.478124 6779 consensus_queue.cc:237] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.483742 6532 tablet_bootstrap.cc:654] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.484965 6879 tablet_bootstrap.cc:492] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.485208 6879 ts_tablet_manager.cc:1403] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.002s
I20260806 07:58:30.485986 6531 tablet_bootstrap.cc:492] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.486181 6531 ts_tablet_manager.cc:1403] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.006s
I20260806 07:58:30.486624 6879 raft_consensus.cc:359] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.486912 6879 raft_consensus.cc:385] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.487025 6879 raft_consensus.cc:740] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.487015 6531 raft_consensus.cc:359] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.487231 6531 raft_consensus.cc:385] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.487308 6531 raft_consensus.cc:740] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.487370 6879 consensus_queue.cc:260] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.487643 6531 consensus_queue.cc:260] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.488005 6879 ts_tablet_manager.cc:1434] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 07:58:30.488458 6885 raft_consensus.cc:493] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.488613 6885 raft_consensus.cc:515] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.489153 6531 ts_tablet_manager.cc:1434] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:30.489293 6885 leader_election.cc:290] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.489440 6531 tablet_bootstrap.cc:492] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.489630 6648 ts_tablet_manager.cc:1929] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.489806 6648 log.cc:1199] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/4f3dc4525dae4b5f83c33b0be1ac01f6
I20260806 07:58:30.490368 6648 ts_tablet_manager.cc:1950] T 4f3dc4525dae4b5f83c33b0be1ac01f6 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.491259 6879 tablet_bootstrap.cc:492] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.488391 6755 ts_tablet_manager.cc:1916] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.490746 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc52165692848699c405d3e54739402" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.493196 6879 tablet_bootstrap.cc:654] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:30.493650 6113 leader_election.cc:343] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.490262 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc52165692848699c405d3e54739402" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
W20260806 07:58:30.497494 6114 leader_election.cc:343] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.497684 6114 leader_election.cc:304] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:30.498050 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 4f3dc4525dae4b5f83c33b0be1ac01f6 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.498270 6544 raft_consensus.cc:2749] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.500207 6531 tablet_bootstrap.cc:654] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.501879 6648 tablet_replica.cc:333] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.502619 6648 raft_consensus.cc:2243] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.503670 6648 raft_consensus.cc:2272] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.504896 6532 tablet_bootstrap.cc:492] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.505159 6532 ts_tablet_manager.cc:1403] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.005s
I20260806 07:58:30.506502 6532 raft_consensus.cc:359] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.506775 6532 raft_consensus.cc:385] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.506927 6532 raft_consensus.cc:740] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.507292 6532 consensus_queue.cc:260] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.508461 6014 catalog_manager.cc:5697] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.508466 6532 ts_tablet_manager.cc:1434] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:30.511790 6532 tablet_bootstrap.cc:492] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.508708 6779 raft_consensus.cc:493] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.512046 6779 raft_consensus.cc:515] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.512805 6779 leader_election.cc:290] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.513173 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef6e56fda745a3aa131e3db6a0d13d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.513408 6315 raft_consensus.cc:2468] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.514024 6113 leader_election.cc:304] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.514142 6648 ts_tablet_manager.cc:1916] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.514297 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef6e56fda745a3aa131e3db6a0d13d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.514513 6779 raft_consensus.cc:2804] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.514626 6450 raft_consensus.cc:2468] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.514742 6779 raft_consensus.cc:493] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.514959 6779 raft_consensus.cc:3060] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.517076 6531 tablet_bootstrap.cc:492] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.517310 6531 ts_tablet_manager.cc:1403] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20260806 07:58:30.518347 6531 raft_consensus.cc:359] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.518553 6531 raft_consensus.cc:385] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.518656 6531 raft_consensus.cc:740] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.518950 6531 consensus_queue.cc:260] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.519556 6531 ts_tablet_manager.cc:1434] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 07:58:30.519886 6531 tablet_bootstrap.cc:492] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.520263 6755 ts_tablet_manager.cc:1929] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.522763 6879 tablet_bootstrap.cc:492] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.522974 6879 ts_tablet_manager.cc:1403] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.004s
I20260806 07:58:30.523101 6532 tablet_bootstrap.cc:654] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.523779 6879 raft_consensus.cc:359] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.523952 6879 raft_consensus.cc:385] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.524044 6879 raft_consensus.cc:740] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.524252 6879 consensus_queue.cc:260] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.524703 6531 tablet_bootstrap.cc:654] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.524766 6879 ts_tablet_manager.cc:1434] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:30.526638 6532 tablet_bootstrap.cc:492] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.527127 6755 log.cc:1199] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/f054c628591a4ef7ab8d607abd77a984
I20260806 07:58:30.527392 6779 raft_consensus.cc:515] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.527314 6532 ts_tablet_manager.cc:1403] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260806 07:58:30.528375 6879 tablet_bootstrap.cc:492] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.528618 6532 raft_consensus.cc:359] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.529560 6531 tablet_bootstrap.cc:492] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.529695 6532 raft_consensus.cc:385] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.529853 6532 raft_consensus.cc:740] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.530192 6532 consensus_queue.cc:260] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.530306 6779 leader_election.cc:290] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.530679 6879 tablet_bootstrap.cc:654] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.531036 6532 ts_tablet_manager.cc:1434] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:30.531435 6532 tablet_bootstrap.cc:492] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.529800 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef6e56fda745a3aa131e3db6a0d13d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.533816 6315 raft_consensus.cc:3060] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.535252 6755 ts_tablet_manager.cc:1950] T f054c628591a4ef7ab8d607abd77a984 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.536298 6755 tablet_replica.cc:333] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.531112 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eef6e56fda745a3aa131e3db6a0d13d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.533823 6532 tablet_bootstrap.cc:654] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.536863 6450 raft_consensus.cc:3060] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.533213 6623 raft_consensus.cc:493] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.537853 6623 raft_consensus.cc:515] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.536891 6531 ts_tablet_manager.cc:1403] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.004s
I20260806 07:58:30.538661 6623 leader_election.cc:290] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079)
I20260806 07:58:30.539449 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6d749166f540a0a83818c2ef5a00e5" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.539645 6318 raft_consensus.cc:2468] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.539788 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet f054c628591a4ef7ab8d607abd77a984 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.540254 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6d749166f540a0a83818c2ef5a00e5" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.540342 6755 raft_consensus.cc:2243] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.540462 6179 raft_consensus.cc:2468] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.540556 6755 raft_consensus.cc:2272] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.541107 6387 leader_election.cc:304] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.542017 6623 raft_consensus.cc:2804] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.542156 6623 raft_consensus.cc:493] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.542258 6623 raft_consensus.cc:3060] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.545044 6450 raft_consensus.cc:2468] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.545998 6114 leader_election.cc:304] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.546350 6779 raft_consensus.cc:2804] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.546567 6779 raft_consensus.cc:697] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.546846 6779 consensus_queue.cc:237] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.546675 6315 raft_consensus.cc:2468] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.548950 6755 ts_tablet_manager.cc:1916] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.549337 6623 raft_consensus.cc:515] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.550729 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6d749166f540a0a83818c2ef5a00e5" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.551029 6315 raft_consensus.cc:3060] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.551472 6592 raft_consensus.cc:493] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.551683 6592 raft_consensus.cc:515] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.552539 6592 leader_election.cc:290] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.553047 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2746179ffc5641c393439e5292ac0b69" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.553323 6623 leader_election.cc:290] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079)
I20260806 07:58:30.553292 6450 raft_consensus.cc:2468] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:30.553332 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2746179ffc5641c393439e5292ac0b69" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.553563 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6d749166f540a0a83818c2ef5a00e5" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.553781 6177 raft_consensus.cc:3060] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.553777 6250 leader_election.cc:304] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
W20260806 07:58:30.554133 6252 leader_election.cc:343] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.554563 6592 raft_consensus.cc:2804] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.554683 6592 raft_consensus.cc:493] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.554764 6592 raft_consensus.cc:3060] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.556109 6315 raft_consensus.cc:2468] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.556622 6384 leader_election.cc:304] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:30.556937 6623 raft_consensus.cc:2804] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.556929 6177 raft_consensus.cc:2468] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.557082 6623 raft_consensus.cc:697] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:30.557411 6623 consensus_queue.cc:237] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } }
I20260806 07:58:30.558761 6592 raft_consensus.cc:515] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.559044 6532 tablet_bootstrap.cc:492] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.559237 6532 ts_tablet_manager.cc:1403] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.003s
I20260806 07:58:30.559178 6531 raft_consensus.cc:359] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.559381 6531 raft_consensus.cc:385] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.559466 6531 raft_consensus.cc:740] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.560288 6531 consensus_queue.cc:260] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.560463 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2746179ffc5641c393439e5292ac0b69" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.560813 6531 ts_tablet_manager.cc:1434] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20260806 07:58:30.561175 6531 tablet_bootstrap.cc:492] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
W20260806 07:58:30.561216 6252 leader_election.cc:343] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.562127 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2746179ffc5641c393439e5292ac0b69" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.562331 6450 raft_consensus.cc:3060] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.562695 6532 raft_consensus.cc:359] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.562878 6532 raft_consensus.cc:385] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.562969 6532 raft_consensus.cc:740] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.563251 6532 consensus_queue.cc:260] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.565227 6532 ts_tablet_manager.cc:1434] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 07:58:30.565297 6450 raft_consensus.cc:2468] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:30.565624 6532 tablet_bootstrap.cc:492] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.565948 6648 ts_tablet_manager.cc:1929] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.566018 6250 leader_election.cc:304] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:30.566097 6648 log.cc:1199] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/de9192eb384944ba9836187ba27edb38
I20260806 07:58:30.566552 6648 ts_tablet_manager.cc:1950] T de9192eb384944ba9836187ba27edb38 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.566578 6718 raft_consensus.cc:2804] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.566552 6592 leader_election.cc:290] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.567869 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet de9192eb384944ba9836187ba27edb38 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.568646 6879 tablet_bootstrap.cc:492] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.568811 6879 ts_tablet_manager.cc:1403] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.003s
I20260806 07:58:30.569609 6879 raft_consensus.cc:359] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.569825 6879 raft_consensus.cc:385] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.569890 6879 raft_consensus.cc:740] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.570217 6879 consensus_queue.cc:260] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.572326 6879 ts_tablet_manager.cc:1434] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:30.572645 6879 tablet_bootstrap.cc:492] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.574901 6879 tablet_bootstrap.cc:654] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.576216 6648 tablet_replica.cc:333] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.576573 6648 raft_consensus.cc:2243] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.576930 6648 raft_consensus.cc:2272] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.576675 6015 catalog_manager.cc:5697] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.578095 6648 ts_tablet_manager.cc:1916] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.583506 6531 tablet_bootstrap.cc:654] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.585515 6532 tablet_bootstrap.cc:654] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.589359 6718 raft_consensus.cc:697] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:30.589787 6718 consensus_queue.cc:237] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.592167 6755 ts_tablet_manager.cc:1929] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.593603 6755 log.cc:1199] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/b5fe3aedfb004f9185b6307e0c087b63
I20260806 07:58:30.596724 6879 tablet_bootstrap.cc:492] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.596943 6879 ts_tablet_manager.cc:1403] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.004s
I20260806 07:58:30.597878 6648 ts_tablet_manager.cc:1929] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.598037 6648 log.cc:1199] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/33b87b2e51024e3d8311bbac25ae573b
I20260806 07:58:30.598450 6648 ts_tablet_manager.cc:1950] T 33b87b2e51024e3d8311bbac25ae573b P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.598789 6879 raft_consensus.cc:359] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.599472 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 33b87b2e51024e3d8311bbac25ae573b (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.599771 6879 raft_consensus.cc:385] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.599977 6879 raft_consensus.cc:740] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.600520 6879 consensus_queue.cc:260] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.601397 6879 ts_tablet_manager.cc:1434] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 07:58:30.601933 6879 tablet_bootstrap.cc:492] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.602262 6648 tablet_replica.cc:333] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.602483 6648 raft_consensus.cc:2243] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.604223 6879 tablet_bootstrap.cc:654] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.605413 6648 raft_consensus.cc:2272] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.606370 6648 ts_tablet_manager.cc:1916] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.595541 6532 tablet_bootstrap.cc:492] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.606714 6532 ts_tablet_manager.cc:1403] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.003s
I20260806 07:58:30.606959 6755 ts_tablet_manager.cc:1950] T b5fe3aedfb004f9185b6307e0c087b63 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.610121 6718 raft_consensus.cc:493] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.610706 6755 tablet_replica.cc:333] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.610955 6755 raft_consensus.cc:2243] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.610930 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet b5fe3aedfb004f9185b6307e0c087b63 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.611137 6755 raft_consensus.cc:2272] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.610371 6718 raft_consensus.cc:515] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.611913 6755 ts_tablet_manager.cc:1916] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.612452 6718 leader_election.cc:290] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.612618 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31bd22c7f6e47c595747dec77a8cc70" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.613315 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31bd22c7f6e47c595747dec77a8cc70" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
W20260806 07:58:30.614019 6252 leader_election.cc:343] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.614092 6016 catalog_manager.cc:5697] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
W20260806 07:58:30.614584 6250 leader_election.cc:343] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.614701 6250 leader_election.cc:304] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:30.617266 6718 raft_consensus.cc:2749] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.617496 6648 ts_tablet_manager.cc:1929] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.617687 6648 log.cc:1199] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/b5fe3aedfb004f9185b6307e0c087b63
I20260806 07:58:30.618213 6648 ts_tablet_manager.cc:1950] T b5fe3aedfb004f9185b6307e0c087b63 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.618546 6755 ts_tablet_manager.cc:1929] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.618705 6755 log.cc:1199] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/33b87b2e51024e3d8311bbac25ae573b
I20260806 07:58:30.620011 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.620710 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.621004 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.621104 6014 catalog_manager.cc:5697] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.621779 6531 tablet_bootstrap.cc:492] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.621939 6531 ts_tablet_manager.cc:1403] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.000s
I20260806 07:58:30.622473 6879 tablet_bootstrap.cc:492] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.622803 6531 raft_consensus.cc:359] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.622998 6531 raft_consensus.cc:385] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.623101 6531 raft_consensus.cc:740] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.623390 6531 consensus_queue.cc:260] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.623940 6648 tablet_replica.cc:333] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.624207 6648 raft_consensus.cc:2243] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.625151 6648 raft_consensus.cc:2272] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.625447 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet b5fe3aedfb004f9185b6307e0c087b63 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.626339 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":164,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.627017 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.627377 6648 ts_tablet_manager.cc:1916] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.627386 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.627766 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.628159 6879 ts_tablet_manager.cc:1403] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.003s
I20260806 07:58:30.628180 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.629422 6879 raft_consensus.cc:359] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.629730 6879 raft_consensus.cc:385] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.629839 6879 raft_consensus.cc:740] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.630173 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.630175 6879 consensus_queue.cc:260] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.630497 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.630868 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.630947 6879 ts_tablet_manager.cc:1434] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:30.631325 6879 tablet_bootstrap.cc:492] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.631533 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.632463 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.632920 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.633489 6531 ts_tablet_manager.cc:1434] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:30.633605 6879 tablet_bootstrap.cc:654] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.637019 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.003s sys 0.000s Metrics: {"bytes_written":490,"delete_count":0,"lbm_write_time_us":190,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:30.637712 6531 tablet_bootstrap.cc:492] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.639676 6531 tablet_bootstrap.cc:654] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.640440 6879 tablet_bootstrap.cc:492] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.640676 6879 ts_tablet_manager.cc:1403] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.004s
I20260806 07:58:30.641271 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.641618 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.641909 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.641916 6879 raft_consensus.cc:359] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.642107 6879 raft_consensus.cc:385] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.642206 6879 raft_consensus.cc:740] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.642233 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.642524 6879 consensus_queue.cc:260] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.643064 6879 ts_tablet_manager.cc:1434] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 07:58:30.643324 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.643621 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.643857 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.643936 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.002s sys 0.001s Metrics: {"bytes_written":498,"delete_count":0,"lbm_write_time_us":203,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:30.644166 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.292725
I20260806 07:58:30.644263 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.644475 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.292725
I20260806 07:58:30.644728 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.292725
I20260806 07:58:30.644834 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.645090 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.645166 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.645388 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.645365 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.645684 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.645797 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.646775 6879 tablet_bootstrap.cc:492] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.650286 6755 ts_tablet_manager.cc:1950] T 33b87b2e51024e3d8311bbac25ae573b P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.619244 6532 raft_consensus.cc:359] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.650692 6532 raft_consensus.cc:385] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.650823 6532 raft_consensus.cc:740] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.650929 6879 tablet_bootstrap.cc:654] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.651154 6532 consensus_queue.cc:260] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.651700 6532 ts_tablet_manager.cc:1434] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.045s user 0.001s sys 0.001s
I20260806 07:58:30.652108 6532 tablet_bootstrap.cc:492] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.652207 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 33b87b2e51024e3d8311bbac25ae573b (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.653385 6755 tablet_replica.cc:333] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.654639 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:30.655110 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.656229 6532 tablet_bootstrap.cc:654] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.656674 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.003s sys 0.002s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":191,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.658414 6879 tablet_bootstrap.cc:492] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.658221 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.005s sys 0.000s Metrics: {"bytes_written":472,"delete_count":0,"lbm_write_time_us":257,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:30.658910 6879 ts_tablet_manager.cc:1403] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260806 07:58:30.659823 6879 raft_consensus.cc:359] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.659933 6648 ts_tablet_manager.cc:1929] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.660128 6879 raft_consensus.cc:385] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.660277 6648 log.cc:1199] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/f054c628591a4ef7ab8d607abd77a984
I20260806 07:58:30.660329 6879 raft_consensus.cc:740] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.660722 6879 consensus_queue.cc:260] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.660953 6648 ts_tablet_manager.cc:1950] T f054c628591a4ef7ab8d607abd77a984 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.662165 6879 ts_tablet_manager.cc:1434] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:30.663467 6879 tablet_bootstrap.cc:492] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.666343 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet f054c628591a4ef7ab8d607abd77a984 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.666839 6648 tablet_replica.cc:333] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.667122 6648 raft_consensus.cc:2243] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.667407 6648 raft_consensus.cc:2272] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.668532 6779 raft_consensus.cc:493] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.668737 6779 raft_consensus.cc:515] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.669432 6779 leader_election.cc:290] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.669649 6879 tablet_bootstrap.cc:654] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.670044 6755 raft_consensus.cc:2243] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.670287 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e7cd5a08d94c519285f18f66c01f0d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.670606 6315 raft_consensus.cc:2468] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.671113 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e7cd5a08d94c519285f18f66c01f0d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.672122 6113 leader_election.cc:304] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
W20260806 07:58:30.672875 6114 leader_election.cc:343] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.673444 6779 raft_consensus.cc:2804] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.673561 6779 raft_consensus.cc:493] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.673935 6779 raft_consensus.cc:3060] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.674818 6648 ts_tablet_manager.cc:1916] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.678910 6879 tablet_bootstrap.cc:492] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.679087 6879 ts_tablet_manager.cc:1403] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.006s
I20260806 07:58:30.679986 6879 raft_consensus.cc:359] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.680236 6879 raft_consensus.cc:385] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.680346 6879 raft_consensus.cc:740] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.680584 6879 consensus_queue.cc:260] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.681272 6879 ts_tablet_manager.cc:1434] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:30.682438 6879 tablet_bootstrap.cc:492] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.682538 6531 tablet_bootstrap.cc:492] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.682730 6531 ts_tablet_manager.cc:1403] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.045s user 0.005s sys 0.011s
I20260806 07:58:30.683586 6531 raft_consensus.cc:359] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.683813 6531 raft_consensus.cc:385] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.683885 6531 raft_consensus.cc:740] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.684226 6531 consensus_queue.cc:260] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.684993 6879 tablet_bootstrap.cc:654] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.688833 6531 ts_tablet_manager.cc:1434] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 07:58:30.690435 6779 raft_consensus.cc:515] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.692855 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.047s user 0.006s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":768,"cfile_init":2,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":56,"lbm_read_time_us":241,"lbm_reads_lt_1ms":12,"lbm_write_time_us":181,"lbm_writes_lt_1ms":4,"mutex_wait_us":15700,"reinsert_count":0,"update_count":4}
I20260806 07:58:30.693529 6531 tablet_bootstrap.cc:492] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.694483 6879 tablet_bootstrap.cc:492] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.694691 6879 ts_tablet_manager.cc:1403] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260806 07:58:30.695029 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e7cd5a08d94c519285f18f66c01f0d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.695403 6315 raft_consensus.cc:3060] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.695595 6879 raft_consensus.cc:359] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.695815 6879 raft_consensus.cc:385] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.695888 6879 raft_consensus.cc:740] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.696257 6531 tablet_bootstrap.cc:654] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.696202 6879 consensus_queue.cc:260] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.697516 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.013s sys 0.000s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":5944,"cfile_init":6,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":1911,"dirs.run_cpu_time_us":129,"dirs.run_wall_time_us":575,"lbm_read_time_us":621,"lbm_reads_lt_1ms":33,"lbm_write_time_us":451,"lbm_writes_lt_1ms":9,"mutex_wait_us":11602,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":23372,"threads_started":1,"update_count":4}
I20260806 07:58:30.698421 6879 ts_tablet_manager.cc:1434] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 07:58:30.698841 6879 tablet_bootstrap.cc:492] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.699340 6648 ts_tablet_manager.cc:1929] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.699460 6648 log.cc:1199] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/fd1e9ae4d73d46d99f50b701d63d4f18
I20260806 07:58:30.699842 6315 raft_consensus.cc:2468] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.700009 6648 ts_tablet_manager.cc:1950] T fd1e9ae4d73d46d99f50b701d63d4f18 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.700702 6113 leader_election.cc:304] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.701112 6583 raft_consensus.cc:2804] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.701145 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet fd1e9ae4d73d46d99f50b701d63d4f18 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.701288 6583 raft_consensus.cc:697] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.701529 6879 tablet_bootstrap.cc:654] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.701774 6648 tablet_replica.cc:333] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.702021 6583 consensus_queue.cc:237] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:30.705148 6114 consensus_peers.cc:597] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7d68050d52894d868b86370cd5e449b8. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:30.706686 6779 leader_election.cc:290] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.670377 6755 raft_consensus.cc:2272] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.708526 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e7cd5a08d94c519285f18f66c01f0d" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.709901 6755 ts_tablet_manager.cc:1916] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.710136 6015 catalog_manager.cc:5697] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.713338 6879 tablet_bootstrap.cc:492] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.713502 6879 ts_tablet_manager.cc:1403] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260806 07:58:30.714322 6879 raft_consensus.cc:359] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.714490 6879 raft_consensus.cc:385] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.714587 6879 raft_consensus.cc:740] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.714802 6879 consensus_queue.cc:260] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.702023 6648 raft_consensus.cc:2243] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.715444 6879 ts_tablet_manager.cc:1434] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:30.715531 6648 raft_consensus.cc:2272] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.715776 6879 tablet_bootstrap.cc:492] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:30.716987 6532 tablet_bootstrap.cc:492] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.717202 6532 ts_tablet_manager.cc:1403] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.065s user 0.026s sys 0.022s
I20260806 07:58:30.717192 6648 ts_tablet_manager.cc:1916] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.717674 6879 tablet_bootstrap.cc:654] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.718334 6532 raft_consensus.cc:359] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.718521 6532 raft_consensus.cc:385] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.718618 6532 raft_consensus.cc:740] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.718879 6532 consensus_queue.cc:260] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.719141 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.719525 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.719843 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.720144 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.720773 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.721087 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.721372 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.721962 6718 raft_consensus.cc:493] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.722131 6718 raft_consensus.cc:515] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.722851 6718 leader_election.cc:290] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.723363 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f80b3d857bd4e62b71bdd63157e8872" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.723590 6450 raft_consensus.cc:2468] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:30.723618 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f80b3d857bd4e62b71bdd63157e8872" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.723835 6177 raft_consensus.cc:2468] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
I20260806 07:58:30.724108 6250 leader_election.cc:304] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:30.724627 6718 raft_consensus.cc:2804] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.724710 6718 raft_consensus.cc:493] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.724802 6718 raft_consensus.cc:3060] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.725725 6648 ts_tablet_manager.cc:1929] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.725881 6648 log.cc:1199] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/43e15cc19276474a8b160d7959d8bf89
I20260806 07:58:30.726336 6648 ts_tablet_manager.cc:1950] T 43e15cc19276474a8b160d7959d8bf89 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.727708 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 43e15cc19276474a8b160d7959d8bf89 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.727896 6718 raft_consensus.cc:515] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.729169 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.729233 6718 leader_election.cc:290] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.729548 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.729777 6532 ts_tablet_manager.cc:1434] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260806 07:58:30.729825 6779 raft_consensus.cc:493] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.729859 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.730011 6779 raft_consensus.cc:515] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.730154 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.730548 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f80b3d857bd4e62b71bdd63157e8872" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.730772 6450 raft_consensus.cc:3060] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.730805 6779 leader_election.cc:290] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
W20260806 07:58:30.731499 6114 leader_election.cc:343] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.732215 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.732276 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.732509 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.732744 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260806 07:58:30.733139 6114 leader_election.cc:343] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.733459 6532 tablet_bootstrap.cc:492] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.733376 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.734160 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.014s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":171,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
W20260806 07:58:30.734498 6113 leader_election.cc:343] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.734691 6113 leader_election.cc:304] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:30.735944 6779 raft_consensus.cc:2749] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.736433 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.736753 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.737056 6450 raft_consensus.cc:2468] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:30.737133 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.737396 6531 tablet_bootstrap.cc:492] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.737632 6531 ts_tablet_manager.cc:1403] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.006s
I20260806 07:58:30.737746 6250 leader_election.cc:304] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:30.738130 6718 raft_consensus.cc:2804] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.738897 6718 raft_consensus.cc:697] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:30.739187 6718 consensus_queue.cc:237] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.740056 6755 ts_tablet_manager.cc:1929] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.740298 6755 log.cc:1199] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/fd1e9ae4d73d46d99f50b701d63d4f18
I20260806 07:58:30.741262 6755 ts_tablet_manager.cc:1950] T fd1e9ae4d73d46d99f50b701d63d4f18 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.741753 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.011s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":222,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.742635 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f80b3d857bd4e62b71bdd63157e8872" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.742954 6177 raft_consensus.cc:3060] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.743656 6531 raft_consensus.cc:359] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.743871 6531 raft_consensus.cc:385] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.745260 6531 raft_consensus.cc:740] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.745816 6531 consensus_queue.cc:260] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.747279 6779 raft_consensus.cc:493] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.747531 6779 raft_consensus.cc:515] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.748353 6779 leader_election.cc:290] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.749225 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9aa031a25104549905f7a7e1007187f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:30.750070 6113 leader_election.cc:343] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.750540 6177 raft_consensus.cc:2468] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:30.750849 6531 ts_tablet_manager.cc:1434] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260806 07:58:30.752966 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9aa031a25104549905f7a7e1007187f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.754060 6532 tablet_bootstrap.cc:654] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:30.754245 6114 leader_election.cc:343] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.754580 6114 leader_election.cc:304] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:30.755148 6779 raft_consensus.cc:2749] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.755962 6648 tablet_replica.cc:333] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.756371 6648 raft_consensus.cc:2243] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:30.756947 6648 raft_consensus.cc:2272] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.758528 6648 ts_tablet_manager.cc:1916] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.758654 6531 tablet_bootstrap.cc:492] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.759652 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.006s sys 0.004s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":788,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":2165,"lbm_read_time_us":211,"lbm_reads_lt_1ms":8,"lbm_write_time_us":241,"lbm_writes_lt_1ms":4,"mutex_wait_us":5005,"reinsert_count":0,"update_count":5}
I20260806 07:58:30.761518 6755 tablet_replica.cc:333] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.761783 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet fd1e9ae4d73d46d99f50b701d63d4f18 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.761855 6755 raft_consensus.cc:2243] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.762055 6879 tablet_bootstrap.cc:492] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:30.762065 6755 raft_consensus.cc:2272] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.762346 6879 ts_tablet_manager.cc:1403] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.047s user 0.011s sys 0.021s
I20260806 07:58:30.763221 6755 ts_tablet_manager.cc:1916] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.763571 6879 raft_consensus.cc:359] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.763948 6879 raft_consensus.cc:385] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.764094 6879 raft_consensus.cc:740] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.764407 6532 tablet_bootstrap.cc:492] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:30.764559 6879 consensus_queue.cc:260] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.764863 6532 ts_tablet_manager.cc:1403] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.000s
I20260806 07:58:30.766283 6532 raft_consensus.cc:359] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.766780 6532 raft_consensus.cc:385] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.766960 6879 ts_tablet_manager.cc:1434] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 07:58:30.766955 6532 raft_consensus.cc:740] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.767633 6532 consensus_queue.cc:260] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.768635 6532 ts_tablet_manager.cc:1434] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 07:58:30.770490 6531 tablet_bootstrap.cc:654] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.771101 6016 catalog_manager.cc:5697] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.776537 6755 ts_tablet_manager.cc:1929] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.776695 6755 log.cc:1199] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/39fcd48f42fc4b0a9edc04adcc8364a0
I20260806 07:58:30.777727 6779 raft_consensus.cc:493] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.778033 6779 raft_consensus.cc:515] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.778363 6648 ts_tablet_manager.cc:1929] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:30.778505 6648 log.cc:1199] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/7d68050d52894d868b86370cd5e449b8
I20260806 07:58:30.778954 6779 leader_election.cc:290] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.779043 6648 ts_tablet_manager.cc:1950] T 7d68050d52894d868b86370cd5e449b8 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:30.779840 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1a3a6825393442a8fa5c96a230afa26" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.780030 6648 tablet_replica.cc:333] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:30.780081 6315 raft_consensus.cc:2468] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.780256 6648 raft_consensus.cc:2243] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.780445 6648 raft_consensus.cc:2272] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.780735 6755 ts_tablet_manager.cc:1950] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:30.780850 6113 leader_election.cc:304] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.781462 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 7d68050d52894d868b86370cd5e449b8 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.782341 6779 raft_consensus.cc:2804] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.782434 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.782473 6779 raft_consensus.cc:493] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.782624 6779 raft_consensus.cc:3060] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.782819 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 39fcd48f42fc4b0a9edc04adcc8364a0 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:30.782861 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.783337 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.783720 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.784061 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.784534 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.784926 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.785189 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.785486 6755 tablet_replica.cc:333] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:30.785705 6755 raft_consensus.cc:2243] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:30.786280 6755 raft_consensus.cc:2272] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:30.786509 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.786665 6583 raft_consensus.cc:493] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.786881 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.786870 6583 raft_consensus.cc:515] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.787199 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.787262 6779 raft_consensus.cc:515] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.787508 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.787724 6583 leader_election.cc:290] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.787940 6779 leader_election.cc:290] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.787971 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.788275 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.788532 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.788717 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1a3a6825393442a8fa5c96a230afa26" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.788890 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.202864
I20260806 07:58:30.788978 6450 raft_consensus.cc:2468] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.790294 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2582adff5654c49b98f072ddfa3ff49" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.790750 6648 ts_tablet_manager.cc:1916] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.791165 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1a3a6825393442a8fa5c96a230afa26" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.791353 6315 raft_consensus.cc:3060] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.791950 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.009s user 0.003s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":233,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
W20260806 07:58:30.792461 6113 leader_election.cc:343] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.793190 6531 tablet_bootstrap.cc:492] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.793499 6531 ts_tablet_manager.cc:1403] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.009s
I20260806 07:58:30.793931 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.794752 6315 raft_consensus.cc:2468] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.794903 6531 raft_consensus.cc:359] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.795123 6531 raft_consensus.cc:385] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.795231 6531 raft_consensus.cc:740] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.795629 6113 leader_election.cc:304] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.795841 6531 consensus_queue.cc:260] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.796039 6779 raft_consensus.cc:2804] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.796176 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2582adff5654c49b98f072ddfa3ff49" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.796252 6779 raft_consensus.cc:697] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.796442 6450 raft_consensus.cc:2468] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.796607 6779 consensus_queue.cc:237] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.797477 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1a3a6825393442a8fa5c96a230afa26" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.797777 6450 raft_consensus.cc:3060] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.797967 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.798062 6531 ts_tablet_manager.cc:1434] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260806 07:58:30.798492 6531 tablet_bootstrap.cc:492] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.803194 6531 tablet_bootstrap.cc:654] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.804000 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.002s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":253,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.804689 6450 raft_consensus.cc:2468] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.806362 6114 leader_election.cc:304] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.807749 6583 raft_consensus.cc:2804] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.807873 6583 raft_consensus.cc:493] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.807992 6583 raft_consensus.cc:3060] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.809705 6532 tablet_bootstrap.cc:492] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:30.810777 6623 raft_consensus.cc:493] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.811023 6623 raft_consensus.cc:515] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.811924 6623 leader_election.cc:290] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.812374 6532 tablet_bootstrap.cc:654] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.813017 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31bd22c7f6e47c595747dec77a8cc70" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.813570 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31bd22c7f6e47c595747dec77a8cc70" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.813833 6177 raft_consensus.cc:2468] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.815728 6387 leader_election.cc:304] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.816222 6623 raft_consensus.cc:2804] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.816387 6623 raft_consensus.cc:493] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.816329 6315 raft_consensus.cc:2468] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.816506 6623 raft_consensus.cc:3060] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.819895 6623 raft_consensus.cc:515] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.822028 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31bd22c7f6e47c595747dec77a8cc70" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.822357 6177 raft_consensus.cc:3060] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.824026 6531 tablet_bootstrap.cc:492] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:30.824229 6531 ts_tablet_manager.cc:1403] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.017s
I20260806 07:58:30.824651 6568 raft_consensus.cc:493] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.824872 6568 raft_consensus.cc:515] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.825364 6531 raft_consensus.cc:359] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.825553 6531 raft_consensus.cc:385] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:30.825755 6531 raft_consensus.cc:740] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:30.825831 6583 raft_consensus.cc:515] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.826022 6531 consensus_queue.cc:260] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.826522 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc52165692848699c405d3e54739402" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.826319 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.018s sys 0.002s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5703,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":665,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":1614,"lbm_read_time_us":561,"lbm_reads_lt_1ms":27,"lbm_write_time_us":341,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":11435904,"thread_start_us":7028,"threads_started":1,"update_count":2}
I20260806 07:58:30.826992 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2582adff5654c49b98f072ddfa3ff49" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.827317 6568 leader_election.cc:290] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
W20260806 07:58:30.827325 6384 leader_election.cc:343] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
W20260806 07:58:30.827596 6113 leader_election.cc:343] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.827961 6583 leader_election.cc:290] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.829175 6177 raft_consensus.cc:2468] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.829754 6623 leader_election.cc:290] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.830889 6387 leader_election.cc:304] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.831168 6531 ts_tablet_manager.cc:1434] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 07:58:30.831616 6531 tablet_bootstrap.cc:492] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:30.831702 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c31bd22c7f6e47c595747dec77a8cc70" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.831892 6318 raft_consensus.cc:3060] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.832816 6016 catalog_manager.cc:5697] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.834218 6531 tablet_bootstrap.cc:654] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:30.835673 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc52165692848699c405d3e54739402" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.835947 6177 raft_consensus.cc:2468] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.836228 6947 raft_consensus.cc:2804] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.836404 6947 raft_consensus.cc:697] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:30.836813 6318 raft_consensus.cc:2468] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.836807 6947 consensus_queue.cc:237] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.837545 6387 leader_election.cc:304] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.838063 6568 raft_consensus.cc:2804] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.838260 6568 raft_consensus.cc:493] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.838402 6568 raft_consensus.cc:3060] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.842193 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.057s user 0.007s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":818,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":44,"lbm_read_time_us":408,"lbm_reads_lt_1ms":8,"lbm_write_time_us":173,"lbm_writes_lt_1ms":4,"mutex_wait_us":7151,"reinsert_count":0,"update_count":6}
I20260806 07:58:30.846753 6755 ts_tablet_manager.cc:1916] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:30.847606 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.848095 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.848495 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.848898 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.849390 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.849776 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.850113 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.853452 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.854048 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.854564 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.854914 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.856268 6568 raft_consensus.cc:515] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.857229 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.857667 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.858122 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.858181 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2582adff5654c49b98f072ddfa3ff49" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.858420 6450 raft_consensus.cc:3060] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.860955 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.012s user 0.003s sys 0.003s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":205,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.862589 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc52165692848699c405d3e54739402" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.862881 6177 raft_consensus.cc:3060] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.863268 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.867478 6177 raft_consensus.cc:2468] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.868392 6387 leader_election.cc:304] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.869076 6568 leader_election.cc:290] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.869333 6568 raft_consensus.cc:2804] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.869472 6568 raft_consensus.cc:697] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:30.869882 6568 consensus_queue.cc:237] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.876102 6450 raft_consensus.cc:2468] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.876565 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc52165692848699c405d3e54739402" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.876804 6114 leader_election.cc:304] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.877228 6779 raft_consensus.cc:493] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.877445 6779 raft_consensus.cc:515] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.878226 6779 leader_election.cc:290] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
W20260806 07:58:30.878376 6384 leader_election.cc:343] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.878433 6779 raft_consensus.cc:2804] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.878646 6779 raft_consensus.cc:697] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.878917 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.879168 6779 consensus_queue.cc:237] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.879405 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.881259 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2746179ffc5641c393439e5292ac0b69" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.883005 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2746179ffc5641c393439e5292ac0b69" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.883365 6450 raft_consensus.cc:2393] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in current term 1: Already voted for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in this term.
I20260806 07:58:30.883914 6016 catalog_manager.cc:5697] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.885583 6114 leader_election.cc:304] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:30.891237 6779 raft_consensus.cc:3060] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.900207 6779 raft_consensus.cc:2749] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:30.900804 6779 raft_consensus.cc:493] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.901124 6779 raft_consensus.cc:515] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.903254 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:30.904278 6113 leader_election.cc:343] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.905504 6779 leader_election.cc:290] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.906019 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.906322 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.906543 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.906833 6450 raft_consensus.cc:2468] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.907047 6623 raft_consensus.cc:493] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.907289 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.003s sys 0.004s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":196,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.907392 6623 raft_consensus.cc:515] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.907730 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.907936 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.908181 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.908545 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.908886 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.909319 6623 leader_election.cc:290] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.909584 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.909842 6114 leader_election.cc:304] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
W20260806 07:58:30.910194 6384 leader_election.cc:343] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.910527 6779 raft_consensus.cc:2804] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.910724 6779 raft_consensus.cc:493] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.910856 6779 raft_consensus.cc:3060] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.911407 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.911732 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.911965 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.912818 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.913096 6177 raft_consensus.cc:2468] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.913700 6387 leader_election.cc:304] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.914299 6623 raft_consensus.cc:2804] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.914716 6623 raft_consensus.cc:493] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.914937 6623 raft_consensus.cc:3060] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.920660 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.200366
I20260806 07:58:30.921031 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.200366
I20260806 07:58:30.921325 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.200366
I20260806 07:58:30.921738 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.922008 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.922524 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:30.923055 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:30.927188 6623 raft_consensus.cc:515] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.932579 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.003s sys 0.004s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":222,"lbm_writes_lt_1ms":4,"mutex_wait_us":17955,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.933163 6583 raft_consensus.cc:493] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.933403 6583 raft_consensus.cc:515] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.935881 6583 leader_election.cc:290] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.936414 6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6182cc1daf4f6ca2c2464b6977cde1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:30.937206 6113 leader_election.cc:343] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.940600 6583 consensus_queue.cc:1048] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:30.937719 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.031s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":30604}
I20260806 07:58:30.943333 6583 raft_consensus.cc:493] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.943598 6583 raft_consensus.cc:515] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.943768 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.026s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":26109}
I20260806 07:58:30.936591 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6182cc1daf4f6ca2c2464b6977cde1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.945039 6583 leader_election.cc:290] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.944972 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85cd55230a604f4c8c22594e2df60249" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.945137 6450 raft_consensus.cc:2468] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.945753 6114 leader_election.cc:304] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.945662 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85cd55230a604f4c8c22594e2df60249" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:30.946017 6451 raft_consensus.cc:2468] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:30.946417 6114 leader_election.cc:304] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.946730 6885 raft_consensus.cc:2804] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.946828 6885 raft_consensus.cc:493] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.946887 6583 raft_consensus.cc:2804] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.946956 6885 raft_consensus.cc:3060] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.947021 6583 raft_consensus.cc:493] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.947186 6583 raft_consensus.cc:3060] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.947968 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.948289 6177 raft_consensus.cc:3060] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.946512 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.037s user 0.003s sys 0.004s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":261,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:30.950981 6315 raft_consensus.cc:3060] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.951381 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.951457 6568 raft_consensus.cc:493] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:30.951674 6568 raft_consensus.cc:515] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:30.951944 6384 leader_election.cc:343] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.953715 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1ce9afe0d394368a97a22242d3e403f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:30.953961 6312 raft_consensus.cc:2468] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.954653 6384 leader_election.cc:304] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:30.956409 6583 raft_consensus.cc:515] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.956696 6568 leader_election.cc:290] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.956923 6568 raft_consensus.cc:2804] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:30.957033 6568 raft_consensus.cc:493] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:30.957167 6568 raft_consensus.cc:3060] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.957170 6885 raft_consensus.cc:515] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.958137 6583 leader_election.cc:290] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.958309 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85cd55230a604f4c8c22594e2df60249" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.958774 6450 raft_consensus.cc:3060] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.958724 6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1ce9afe0d394368a97a22242d3e403f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:30.958986 6179 raft_consensus.cc:2468] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:30.959587 6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6182cc1daf4f6ca2c2464b6977cde1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:30.959858 6451 raft_consensus.cc:3060] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.963379 6450 raft_consensus.cc:2468] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.964862 6451 raft_consensus.cc:2468] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:30.965356 6114 leader_election.cc:304] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:30.965832 6583 raft_consensus.cc:2804] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.965862 6114 leader_election.cc:304] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
W20260806 07:58:30.966089 6113 leader_election.cc:343] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:30.967149 6583 raft_consensus.cc:697] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.967485 6544 raft_consensus.cc:2804] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.969579 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85cd55230a604f4c8c22594e2df60249" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.970387 6623 leader_election.cc:290] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.971813 6583 consensus_queue.cc:237] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.979470 6885 leader_election.cc:290] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:30.979712 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6182cc1daf4f6ca2c2464b6977cde1" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.980561 6177 raft_consensus.cc:2468] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.979238 6568 raft_consensus.cc:515] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.983268 6016 catalog_manager.cc:5697] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:30.984764 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1ce9afe0d394368a97a22242d3e403f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:30.984869 6387 leader_election.cc:304] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:30.985466 6315 raft_consensus.cc:3060] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.985826 6544 raft_consensus.cc:697] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:30.986326 6544 consensus_queue.cc:237] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.991813 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1ce9afe0d394368a97a22242d3e403f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:30.992218 6177 raft_consensus.cc:3060] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:30.989951 6953 raft_consensus.cc:2804] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:30.995874 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.996399 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.996546 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.996930 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.996963 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.997324 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:30.997737 6568 leader_election.cc:290] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:30.997864 6177 raft_consensus.cc:2468] T a1ce9afe0d394368a97a22242d3e403f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:30.998118 6953 raft_consensus.cc:697] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:30.998606 6953 consensus_queue.cc:237] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:30.999895 6648 ts_tablet_manager.cc:1929] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.000823 6387 leader_election.cc:304] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:31.001187 6648 log.cc:1199] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/39fcd48f42fc4b0a9edc04adcc8364a0
I20260806 07:58:31.000341 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.001701 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.002676 6648 ts_tablet_manager.cc:1950] T 39fcd48f42fc4b0a9edc04adcc8364a0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:31.004577 6648 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39230 (request call id 93) took 1115 ms. Trace:
I20260806 07:58:31.005134 6315 raft_consensus.cc:2468] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:31.006050 6648 rpcz_store.cc:276] 0806 07:58:29.888702 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:29.889585 (+ 883us) service_pool.cc:224] Handling call
0806 07:58:30.779949 (+890364us) ts_tablet_manager.cc:1151] Deleting tablet 39fcd48f42fc4b0a9edc04adcc8364a0
0806 07:58:30.779961 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:30.852177 (+ 72216us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:30.999807 (+147630us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:31.004283 (+ 4476us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":890168}
I20260806 07:58:31.006260 6568 raft_consensus.cc:2804] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:31.007004 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 39fcd48f42fc4b0a9edc04adcc8364a0 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:31.007589 6568 raft_consensus.cc:697] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:31.008152 6568 consensus_queue.cc:237] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.009763 6718 consensus_queue.cc:1048] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.017333 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":19958}
I20260806 07:58:31.018044 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":20591}
I20260806 07:58:31.018617 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.006s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":212,"lbm_writes_lt_1ms":4,"mutex_wait_us":12915,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.018446 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.150s user 0.006s sys 0.005s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":846,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":175,"dirs.run_wall_time_us":674,"lbm_read_time_us":209,"lbm_reads_lt_1ms":8,"lbm_write_time_us":244,"lbm_writes_lt_1ms":4,"mutex_wait_us":18586,"reinsert_count":0,"update_count":7}
I20260806 07:58:31.019292 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.098s user 0.016s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5777,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":588,"dirs.run_cpu_time_us":312,"dirs.run_wall_time_us":6228,"lbm_read_time_us":28921,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":26,"lbm_write_time_us":957,"lbm_writes_lt_1ms":9,"mutex_wait_us":7545,"peak_mem_usage":32768,"reinsert_count":0,"update_count":2}
I20260806 07:58:31.020527 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:31.020989 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:31.021488 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.003000
I20260806 07:58:31.022141 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.022979 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.025861 6779 raft_consensus.cc:515] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.037519 6718 raft_consensus.cc:493] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.037865 6718 raft_consensus.cc:515] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.038931 6718 leader_election.cc:290] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.039628 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:31.039894 6450 raft_consensus.cc:2468] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:31.039732 6016 catalog_manager.cc:5697] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:31.040642 6250 leader_election.cc:304] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:31.041170 6718 raft_consensus.cc:2804] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 07:58:31.041316 6718 raft_consensus.cc:493] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:31.041441 6718 raft_consensus.cc:3060] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:58:31.046546 6718 raft_consensus.cc:515] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.047945 6718 leader_election.cc:290] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 2 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.051787 6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:31.052078 6450 raft_consensus.cc:3060] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:58:31.055632 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.056087 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.056443 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.056656 6450 raft_consensus.cc:2468] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 2.
I20260806 07:58:31.056931 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.057802 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.058249 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.058555 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.058641 6250 leader_election.cc:304] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters:
I20260806 07:58:31.061079 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.197929
I20260806 07:58:31.061405 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.197929
I20260806 07:58:31.061750 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.197929
I20260806 07:58:31.062163 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.062417 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.062809 6718 raft_consensus.cc:2804] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 07:58:31.063073 6718 raft_consensus.cc:697] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 2 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:31.063429 6718 consensus_queue.cc:237] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.065680 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.068s user 0.007s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.066329 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.066479 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.066960 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.067379 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.071107 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.071389 6568 consensus_queue.cc:1048] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.081223 6532 tablet_bootstrap.cc:492] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.081447 6532 ts_tablet_manager.cc:1403] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.272s user 0.080s sys 0.108s
I20260806 07:58:31.082680 6532 raft_consensus.cc:359] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.082928 6532 raft_consensus.cc:385] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.083105 6532 raft_consensus.cc:740] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.083436 6532 consensus_queue.cc:260] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.087175 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.001s sys 0.006s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":1695,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":3,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.091001 6015 catalog_manager.cc:5697] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:31.101182 6531 tablet_bootstrap.cc:492] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:31.101514 6531 ts_tablet_manager.cc:1403] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.270s user 0.034s sys 0.066s
I20260806 07:58:31.102427 6568 consensus_queue.cc:1048] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.102828 6531 raft_consensus.cc:359] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.103201 6531 raft_consensus.cc:385] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.103300 6531 raft_consensus.cc:740] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.103717 6531 consensus_queue.cc:260] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.104785 6531 ts_tablet_manager.cc:1434] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 07:58:31.105382 6531 tablet_bootstrap.cc:492] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:31.108743 6531 tablet_bootstrap.cc:654] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.070101 6885 consensus_queue.cc:1048] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.090s
W20260806 07:58:31.071163 6113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.202.194:32979, user_credentials={real_user=slave}} blocked reactor thread for 102552us
I20260806 07:58:31.111008 6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:31.111151 6113 leader_election.cc:343] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
W20260806 07:58:31.111562 6113 leader_election.cc:343] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:31.111809 6015 catalog_manager.cc:5697] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:31.113476 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be96d575a672438994b6c6364180ec84" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:31.117568 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:31.118417 6113 leader_election.cc:343] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:31.118822 6918 consensus_queue.cc:1048] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.119395 6779 leader_election.cc:290] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.119645 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:31.119930 6453 raft_consensus.cc:3060] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.120383 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.120779 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.122385 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.113447 6015 catalog_manager.cc:5697] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:31.123926 6453 raft_consensus.cc:2468] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:31.124215 6015 catalog_manager.cc:5697] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:31.124837 6114 leader_election.cc:304] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:31.125528 6955 raft_consensus.cc:2804] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:31.125772 6955 raft_consensus.cc:697] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:31.126194 6955 consensus_queue.cc:237] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.126752 6918 consensus_queue.cc:1048] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.127205 6451 raft_consensus.cc:1240] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 2 FOLLOWER]: Rejecting Update request from peer 7793159fc1ab4bd1b2a50dbd3b3e5168 for earlier term 1. Current term is 2. Ops: []
W20260806 07:58:31.131582 6114 consensus_peers.cc:597] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:31.138046 6583 consensus_queue.cc:1059] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.138401 6583 raft_consensus.cc:3055] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Stepping down as leader of term 1
I20260806 07:58:31.138504 6583 raft_consensus.cc:740] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:31.138856 6583 consensus_queue.cc:260] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.139213 6583 raft_consensus.cc:3060] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Advancing to term 2
I20260806 07:58:31.149574 6583 consensus_queue.cc:1048] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.167546 6918 consensus_queue.cc:1048] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.172319 6779 consensus_queue.cc:1048] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.176668 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.115s user 0.013s sys 0.001s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5851,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":1342,"lbm_read_time_us":594,"lbm_reads_lt_1ms":27,"lbm_write_time_us":610,"lbm_writes_lt_1ms":9,"mutex_wait_us":11841,"peak_mem_usage":32768,"reinsert_count":0,"update_count":2}
I20260806 07:58:31.179718 6315 raft_consensus.cc:1240] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 2 LEADER]: Rejecting Update request from peer 7793159fc1ab4bd1b2a50dbd3b3e5168 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260806 07:58:31.182339 6015 catalog_manager.cc:5697] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 1 to 2, leader changed from 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193) to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 2 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:31.187922 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":65712}
I20260806 07:58:31.188282 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":65227}
I20260806 07:58:31.194288 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.172s user 0.009s sys 0.001s Metrics: {"cfile_cache_miss":6,"cfile_cache_miss_bytes":1207,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":1205,"dirs.run_cpu_time_us":156,"dirs.run_wall_time_us":2066,"lbm_read_time_us":269,"lbm_reads_lt_1ms":10,"lbm_write_time_us":259,"lbm_writes_lt_1ms":4,"mutex_wait_us":39789,"reinsert_count":0,"update_count":9}
I20260806 07:58:31.195147 6755 ts_tablet_manager.cc:1929] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.195122 6779 consensus_queue.cc:1048] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.195194 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.173136
I20260806 07:58:31.195822 6755 log.cc:1199] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/7d68050d52894d868b86370cd5e449b8
I20260806 07:58:31.199512 6755 ts_tablet_manager.cc:1950] T 7d68050d52894d868b86370cd5e449b8 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.199862 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.078s user 0.003s sys 0.003s Metrics: {"bytes_written":497,"delete_count":0,"lbm_write_time_us":194,"lbm_writes_lt_1ms":4,"mutex_wait_us":55294,"reinsert_count":0,"update_count":4}
I20260806 07:58:31.200917 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.201712 6755 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36458 (request call id 95) took 1316 ms. Trace:
I20260806 07:58:31.203729 6755 rpcz_store.cc:276] 0806 07:58:29.885267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:29.885393 (+ 126us) service_pool.cc:224] Handling call
0806 07:58:30.781850 (+896457us) ts_tablet_manager.cc:1151] Deleting tablet 7d68050d52894d868b86370cd5e449b8
0806 07:58:30.781859 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:31.017354 (+235495us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:31.093446 (+ 76092us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:31.201670 (+108224us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":896163}
I20260806 07:58:31.204443 6755 tablet_replica.cc:333] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.204771 6755 raft_consensus.cc:2243] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.200747 6532 ts_tablet_manager.cc:1434] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.119s user 0.052s sys 0.048s
I20260806 07:58:31.205313 6532 tablet_bootstrap.cc:492] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:31.205476 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 7d68050d52894d868b86370cd5e449b8 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:31.207933 6532 tablet_bootstrap.cc:654] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.209059 6779 consensus_queue.cc:1048] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.209241 6755 raft_consensus.cc:2272] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.210000 6755 ts_tablet_manager.cc:1916] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.211217 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.001s sys 0.004s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":171,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.211793 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.152347
I20260806 07:58:31.212152 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.152347
I20260806 07:58:31.212499 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.152347
I20260806 07:58:31.212841 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.152347
I20260806 07:58:31.213902 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.214197 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.214466 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.216585 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.148s user 0.003s sys 0.003s Metrics: {"bytes_written":534,"delete_count":0,"lbm_write_time_us":162,"lbm_writes_lt_1ms":4,"mutex_wait_us":124300,"reinsert_count":0,"update_count":5}
I20260806 07:58:31.219317 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.220016 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.222153 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.128s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":128203}
I20260806 07:58:31.222110 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.152s user 0.005s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":191,"lbm_writes_lt_1ms":4,"mutex_wait_us":146348,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.226322 6016 catalog_manager.cc:5697] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:31.232216 6532 tablet_bootstrap.cc:492] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.232455 6532 ts_tablet_manager.cc:1403] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.004s
I20260806 07:58:31.233177 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.013s sys 0.000s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":5981,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":185,"lbm_read_time_us":590,"lbm_reads_lt_1ms":27,"lbm_write_time_us":1026,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"update_count":5}
I20260806 07:58:31.233556 6532 raft_consensus.cc:359] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.233819 6532 raft_consensus.cc:385] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.233906 6532 raft_consensus.cc:740] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.238667 6531 tablet_bootstrap.cc:492] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:31.238852 6531 ts_tablet_manager.cc:1403] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.134s user 0.004s sys 0.004s
I20260806 07:58:31.239929 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.239751 6531 raft_consensus.cc:359] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.240099 6531 raft_consensus.cc:385] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.240244 6531 raft_consensus.cc:740] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.240149 6947 consensus_queue.cc:1048] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.240437 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.240773 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.242424 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.243671 6531 consensus_queue.cc:260] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.244135 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.244503 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.244889 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.245924 6531 ts_tablet_manager.cc:1434] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260806 07:58:31.247888 6532 consensus_queue.cc:260] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.248382 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.248952 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.250324 6532 ts_tablet_manager.cc:1434] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.018s user 0.003s sys 0.000s
I20260806 07:58:31.253079 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.008s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.256163 6532 tablet_bootstrap.cc:492] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:31.258561 6532 tablet_bootstrap.cc:654] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.261921 6568 consensus_queue.cc:1048] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.267501 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.005s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":181,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.271593 6755 ts_tablet_manager.cc:1929] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.271785 6755 log.cc:1199] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/43e15cc19276474a8b160d7959d8bf89
I20260806 07:58:31.272322 6755 ts_tablet_manager.cc:1950] T 43e15cc19276474a8b160d7959d8bf89 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.273310 6755 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36458 (request call id 94) took 1388 ms. Trace:
I20260806 07:58:31.273452 6755 rpcz_store.cc:276] 0806 07:58:29.884425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0806 07:58:29.887540 (+ 3115us) service_pool.cc:224] Handling call
0806 07:58:31.204269 (+1316729us) ts_tablet_manager.cc:1151] Deleting tablet 43e15cc19276474a8b160d7959d8bf89
0806 07:58:31.204277 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0806 07:58:31.251007 (+ 46730us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:31.271486 (+ 20479us) tablet_metadata.cc:665] Metadata flushed
0806 07:58:31.273211 (+ 1725us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1316537}
I20260806 07:58:31.274293 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 43e15cc19276474a8b160d7959d8bf89 (table table-2 [id=3fc3653402c94f3e85bec250c0e53983]) successfully deleted
I20260806 07:58:31.291100 6532 tablet_bootstrap.cc:492] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.291301 6532 ts_tablet_manager.cc:1403] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.007s
I20260806 07:58:31.292553 6532 raft_consensus.cc:359] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.292812 6532 raft_consensus.cc:385] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.293007 6532 raft_consensus.cc:740] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.293305 6532 consensus_queue.cc:260] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.294298 6532 ts_tablet_manager.cc:1434] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260806 07:58:31.294677 6532 tablet_bootstrap.cc:492] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:31.297253 6532 tablet_bootstrap.cc:654] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.299351 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.101s user 0.013s sys 0.011s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":24173,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":3,"delta_iterators_relevant":3,"dirs.queue_time_us":4591,"dirs.run_cpu_time_us":322,"dirs.run_wall_time_us":3519,"lbm_read_time_us":816,"lbm_reads_lt_1ms":29,"lbm_write_time_us":423,"lbm_writes_lt_1ms":9,"mutex_wait_us":23791,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":13204,"threads_started":1,"update_count":15}
I20260806 07:58:31.310751 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":65250}
I20260806 07:58:31.311143 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.065s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":64600}
I20260806 07:58:31.311791 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.066s user 0.008s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":247,"lbm_writes_lt_1ms":4,"mutex_wait_us":54079,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.313797 6532 tablet_bootstrap.cc:492] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.314218 6532 ts_tablet_manager.cc:1403] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.012s
I20260806 07:58:31.315318 6532 raft_consensus.cc:359] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.315600 6532 raft_consensus.cc:385] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.315757 6532 raft_consensus.cc:740] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.316136 6532 consensus_queue.cc:260] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.316789 6532 ts_tablet_manager.cc:1434] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:31.317792 6532 tablet_bootstrap.cc:492] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:31.320571 6532 tablet_bootstrap.cc:654] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.326917 6532 tablet_bootstrap.cc:492] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.327169 6532 ts_tablet_manager.cc:1403] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260806 07:58:31.328456 6532 raft_consensus.cc:359] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.328657 6532 raft_consensus.cc:385] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.328797 6532 raft_consensus.cc:740] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.329070 6532 consensus_queue.cc:260] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.329651 6532 ts_tablet_manager.cc:1434] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 07:58:31.329975 6532 tablet_bootstrap.cc:492] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:31.332289 6532 tablet_bootstrap.cc:654] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.336931 6947 raft_consensus.cc:493] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.337111 6947 raft_consensus.cc:515] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.338032 6947 leader_election.cc:290] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:31.339474 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e7cd5a08d94c519285f18f66c01f0d" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:31.340102 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e7cd5a08d94c519285f18f66c01f0d" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:31.341046 6384 leader_election.cc:304] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:31.341531 6947 raft_consensus.cc:2749] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:31.342337 6532 tablet_bootstrap.cc:492] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.342548 6532 ts_tablet_manager.cc:1403] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.002s
I20260806 07:58:31.343101 6718 raft_consensus.cc:493] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.343271 6718 raft_consensus.cc:515] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.343513 6532 raft_consensus.cc:359] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.343731 6532 raft_consensus.cc:385] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.343815 6532 raft_consensus.cc:740] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.344008 6718 leader_election.cc:290] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.344403 6532 consensus_queue.cc:260] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.345206 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:31.345325 6532 ts_tablet_manager.cc:1434] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260806 07:58:31.345377 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b13ae96de4eb42088b187c4efc95802a" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:31.345649 6177 raft_consensus.cc:2393] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:31.346774 6532 tablet_bootstrap.cc:492] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:31.349268 6532 tablet_bootstrap.cc:654] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.350044 6250 leader_election.cc:304] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:31.352108 6718 raft_consensus.cc:3060] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.356791 6718 raft_consensus.cc:2749] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:31.359177 6532 tablet_bootstrap.cc:492] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:31.359392 6532 ts_tablet_manager.cc:1403] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.004s
I20260806 07:58:31.359755 6718 raft_consensus.cc:493] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.359900 6718 raft_consensus.cc:515] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.361537 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6182cc1daf4f6ca2c2464b6977cde1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:31.361579 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6182cc1daf4f6ca2c2464b6977cde1" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:31.361836 6453 raft_consensus.cc:2393] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:31.362102 6532 raft_consensus.cc:359] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.362409 6532 raft_consensus.cc:385] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:31.362587 6532 raft_consensus.cc:740] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:31.363085 6918 raft_consensus.cc:493] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.363013 6532 consensus_queue.cc:260] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.363526 6252 leader_election.cc:304] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:31.363440 6918 raft_consensus.cc:515] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.364559 6964 raft_consensus.cc:3060] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.369990 6918 leader_election.cc:290] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.370148 6947 raft_consensus.cc:493] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.370352 6947 raft_consensus.cc:515] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.370875 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85cd55230a604f4c8c22594e2df60249" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:31.371057 6947 leader_election.cc:290] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:31.371987 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9aa031a25104549905f7a7e1007187f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:31.372961 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85cd55230a604f4c8c22594e2df60249" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:31.373106 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9aa031a25104549905f7a7e1007187f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:31.373219 6453 raft_consensus.cc:2393] T 85cd55230a604f4c8c22594e2df60249 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:31.373332 6177 raft_consensus.cc:2468] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:31.373965 6387 leader_election.cc:304] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:31.374452 6250 leader_election.cc:304] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:31.375072 6568 raft_consensus.cc:2804] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:31.375218 6568 raft_consensus.cc:493] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:31.375308 6568 raft_consensus.cc:3060] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.375888 6312 raft_consensus.cc:3060] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.376199 6964 raft_consensus.cc:2749] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:31.376364 6718 leader_election.cc:290] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.376802 6314 raft_consensus.cc:2468] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:31.377156 6918 raft_consensus.cc:3060] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.379079 6568 raft_consensus.cc:515] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.380319 6918 raft_consensus.cc:2749] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:31.380404 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9aa031a25104549905f7a7e1007187f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:31.380635 6177 raft_consensus.cc:3060] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.381681 6947 consensus_queue.cc:1048] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.382542 6532 ts_tablet_manager.cc:1434] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 07:58:31.384881 6177 raft_consensus.cc:2468] T f9aa031a25104549905f7a7e1007187f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:31.385617 6387 leader_election.cc:304] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:31.386224 6947 raft_consensus.cc:2804] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:31.386385 6947 raft_consensus.cc:697] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:31.386673 6947 consensus_queue.cc:237] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.388753 6955 consensus_queue.cc:1048] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.389321 6314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9aa031a25104549905f7a7e1007187f" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:31.389652 6314 raft_consensus.cc:3060] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.389405 6568 leader_election.cc:290] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:31.394049 6314 raft_consensus.cc:2468] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:31.399751 6955 consensus_queue.cc:1048] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.406147 6623 consensus_queue.cc:1048] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.419399 6623 consensus_queue.cc:1048] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.422493 6623 consensus_queue.cc:1048] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.428498 6955 consensus_queue.cc:1048] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.441910 6015 catalog_manager.cc:5697] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:31.449563 6312 raft_consensus.cc:3060] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.455194 6544 consensus_queue.cc:1048] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 07:58:31.459940 6544 consensus_queue.cc:1048] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.462582 6885 consensus_queue.cc:1048] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.478626 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.479076 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.479211 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.479374 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.479612 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.479748 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.480023 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.480319 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.480527 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.190960
I20260806 07:58:31.481046 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.481178 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.481441 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.481505 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.481757 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.482197 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.482465 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.484513 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.002s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":183,"lbm_writes_lt_1ms":4,"reinsert_count":0,"spinlock_wait_cycles":768,"update_count":1}
I20260806 07:58:31.486089 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.005s sys 0.001s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":225,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:31.487205 6568 consensus_queue.cc:1048] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.490271 6947 consensus_queue.cc:1048] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.502522 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.006s sys 0.001s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":54,"lbm_read_time_us":212,"lbm_reads_lt_1ms":8,"lbm_write_time_us":158,"lbm_writes_lt_1ms":4,"mutex_wait_us":4032,"reinsert_count":0,"update_count":2}
I20260806 07:58:31.518638 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.009s sys 0.003s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":6073,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":284,"lbm_read_time_us":551,"lbm_reads_lt_1ms":27,"lbm_write_time_us":2417,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"mutex_wait_us":3585,"peak_mem_usage":32768,"reinsert_count":0,"spinlock_wait_cycles":36736,"update_count":2}
I20260806 07:58:31.528010 6452 raft_consensus.cc:3060] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.617187 6918 raft_consensus.cc:493] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:31.617378 6918 raft_consensus.cc:515] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:31.618279 6918 leader_election.cc:290] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:31.618865 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:31.618891 6177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6c454c14d784d5f9acb483fc8d26ae3" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:31.619263 6452 raft_consensus.cc:2393] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in this term.
I20260806 07:58:31.619800 6250 leader_election.cc:304] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:31.620180 6918 raft_consensus.cc:3060] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:31.622877 6918 raft_consensus.cc:2749] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:31.658277 6177 raft_consensus.cc:1275] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 2 FOLLOWER]: Refusing update from remote peer 0d9e5266c4e04896a714d37cd67fdcc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260806 07:58:31.658417 6177 pending_rounds.cc:85] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Aborting all ops after (but not including) 0
I20260806 07:58:31.658608 6177 pending_rounds.cc:107] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260806 07:58:31.658766 6177 raft_consensus.cc:2889] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260806 07:58:31.659561 6918 consensus_queue.cc:1048] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.663669 6452 raft_consensus.cc:1275] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 2 FOLLOWER]: Refusing update from remote peer 0d9e5266c4e04896a714d37cd67fdcc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260806 07:58:31.664595 6718 consensus_queue.cc:1048] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.692162 6954 consensus_queue.cc:1048] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.699175 6954 consensus_queue.cc:1048] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.699509 6017 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-3" } modify_external_catalogs: true
I20260806 07:58:31.699759 6017 catalog_manager.cc:2781] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46090:
table { table_name: "table-3" } modify_external_catalogs: true
I20260806 07:58:31.713948 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 5e6d749166f540a0a83818c2ef5a00e5
I20260806 07:58:31.715195 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e6d749166f540a0a83818c2ef5a00e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.715322 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e6d749166f540a0a83818c2ef5a00e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.715622 6755 tablet_replica.cc:333] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.715668 6420 tablet_service.cc:1558] Processing DeleteTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.715852 6755 raft_consensus.cc:2243] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.716001 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.716190 6755 raft_consensus.cc:2272] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.716411 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.716741 7016 tablet_replica.cc:333] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:31.716747 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.716763 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e6d749166f540a0a83818c2ef5a00e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.716974 7016 raft_consensus.cc:2243] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:31.717167 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.717363 6755 ts_tablet_manager.cc:1916] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.717367 7016 raft_consensus.cc:2272] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.717470 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.717792 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.718135 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.718340 7016 ts_tablet_manager.cc:1916] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.722810 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.000s sys 0.004s Metrics: {"bytes_written":982,"delete_count":0,"lbm_write_time_us":211,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:31.723320 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.153943
I20260806 07:58:31.723611 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.153943
I20260806 07:58:31.723930 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.153943
I20260806 07:58:31.724289 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.724506 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.725040 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.725298 6755 ts_tablet_manager.cc:1929] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.725381 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.725463 6755 log.cc:1199] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/5e6d749166f540a0a83818c2ef5a00e5
I20260806 07:58:31.725739 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.726028 6755 ts_tablet_manager.cc:1950] T 5e6d749166f540a0a83818c2ef5a00e5 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.726297 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.726743 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.728215 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 5e6d749166f540a0a83818c2ef5a00e5 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.728760 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.314734
I20260806 07:58:31.728966 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 5eef6e56fda745a3aa131e3db6a0d13d
I20260806 07:58:31.729038 7017 tablet_replica.cc:333] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:31.729125 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.314734
I20260806 07:58:31.729452 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.314734
I20260806 07:58:31.729534 7017 raft_consensus.cc:2243] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.729740 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.729856 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.729908 7017 raft_consensus.cc:2272] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.729965 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 5eef6e56fda745a3aa131e3db6a0d13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.730938 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet d1a3a6825393442a8fa5c96a230afa26
I20260806 07:58:31.730953 7017 ts_tablet_manager.cc:1916] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.731181 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 5eef6e56fda745a3aa131e3db6a0d13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.731513 6755 tablet_replica.cc:333] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.731714 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet d1a3a6825393442a8fa5c96a230afa26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.731905 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 5eef6e56fda745a3aa131e3db6a0d13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.731969 6755 raft_consensus.cc:2243] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.732209 6755 raft_consensus.cc:2272] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.732758 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet d1a3a6825393442a8fa5c96a230afa26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.733265 6755 ts_tablet_manager.cc:1916] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.733821 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:31.734117 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.734462 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet f6c454c14d784d5f9acb483fc8d26ae3
I20260806 07:58:31.735304 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.736362 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet d1a3a6825393442a8fa5c96a230afa26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.736424 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 81e7cd5a08d94c519285f18f66c01f0d
I20260806 07:58:31.736902 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.736958 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet f6c454c14d784d5f9acb483fc8d26ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.737617 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 81e7cd5a08d94c519285f18f66c01f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.738634 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 81e7cd5a08d94c519285f18f66c01f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.738714 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 7f80b3d857bd4e62b71bdd63157e8872
I20260806 07:58:31.739580 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f80b3d857bd4e62b71bdd63157e8872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.739925 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 81e7cd5a08d94c519285f18f66c01f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.740531 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f80b3d857bd4e62b71bdd63157e8872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.740631 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet d2582adff5654c49b98f072ddfa3ff49
I20260806 07:58:31.741163 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f80b3d857bd4e62b71bdd63157e8872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.741251 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet d2582adff5654c49b98f072ddfa3ff49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.741205 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.006s sys 0.000s Metrics: {"bytes_written":982,"delete_count":0,"lbm_write_time_us":184,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":17}
I20260806 07:58:31.742242 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 2746179ffc5641c393439e5292ac0b69
I20260806 07:58:31.742273 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet d2582adff5654c49b98f072ddfa3ff49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.743011 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet d2582adff5654c49b98f072ddfa3ff49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.743026 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 2746179ffc5641c393439e5292ac0b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.743968 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 2746179ffc5641c393439e5292ac0b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.744122 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet b13ae96de4eb42088b187c4efc95802a
I20260806 07:58:31.743991 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.020s user 0.010s sys 0.004s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":6348,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":89,"lbm_read_time_us":1521,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1709,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":8,"peak_mem_usage":32768,"reinsert_count":0,"update_count":17}
I20260806 07:58:31.744872 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet b13ae96de4eb42088b187c4efc95802a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.745040 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 2746179ffc5641c393439e5292ac0b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.745286 7016 ts_tablet_manager.cc:1929] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.745425 7016 log.cc:1199] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/5e6d749166f540a0a83818c2ef5a00e5
I20260806 07:58:31.746176 7016 ts_tablet_manager.cc:1950] T 5e6d749166f540a0a83818c2ef5a00e5 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:31.746447 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet b13ae96de4eb42088b187c4efc95802a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.746716 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 5a6182cc1daf4f6ca2c2464b6977cde1
I20260806 07:58:31.747025 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet b13ae96de4eb42088b187c4efc95802a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.747519 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a6182cc1daf4f6ca2c2464b6977cde1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.747857 7016 tablet_replica.cc:333] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:31.747961 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 5e6d749166f540a0a83818c2ef5a00e5 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.748116 7016 raft_consensus.cc:2243] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.748322 7016 raft_consensus.cc:2272] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.748796 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a6182cc1daf4f6ca2c2464b6977cde1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.749140 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet be96d575a672438994b6c6364180ec84
I20260806 07:58:31.749384 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a6182cc1daf4f6ca2c2464b6977cde1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.749622 7016 ts_tablet_manager.cc:1916] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.749966 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet be96d575a672438994b6c6364180ec84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.750780 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet be96d575a672438994b6c6364180ec84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.751178 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet dbc52165692848699c405d3e54739402
I20260806 07:58:31.751434 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet be96d575a672438994b6c6364180ec84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.752193 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc52165692848699c405d3e54739402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.753116 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc52165692848699c405d3e54739402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.753510 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet c31bd22c7f6e47c595747dec77a8cc70
I20260806 07:58:31.753706 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc52165692848699c405d3e54739402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.753831 6755 ts_tablet_manager.cc:1929] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.754020 6755 log.cc:1199] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/5eef6e56fda745a3aa131e3db6a0d13d
I20260806 07:58:31.754550 6755 ts_tablet_manager.cc:1950] T 5eef6e56fda745a3aa131e3db6a0d13d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.754920 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet c31bd22c7f6e47c595747dec77a8cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.755306 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet f9aa031a25104549905f7a7e1007187f
I20260806 07:58:31.755528 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet c31bd22c7f6e47c595747dec77a8cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.755973 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet c31bd22c7f6e47c595747dec77a8cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.756733 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet f9aa031a25104549905f7a7e1007187f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.757517 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet f9aa031a25104549905f7a7e1007187f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.757910 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet a1ce9afe0d394368a97a22242d3e403f
I20260806 07:58:31.758028 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet f9aa031a25104549905f7a7e1007187f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.758824 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet a1ce9afe0d394368a97a22242d3e403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.759047 7016 ts_tablet_manager.cc:1929] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.759215 7016 log.cc:1199] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/f6c454c14d784d5f9acb483fc8d26ae3
I20260806 07:58:31.759581 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet a1ce9afe0d394368a97a22242d3e403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.759380 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.014s sys 0.003s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":24597,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":86,"lbm_read_time_us":764,"lbm_reads_lt_1ms":27,"lbm_write_time_us":566,"lbm_writes_lt_1ms":9,"mutex_wait_us":11774,"peak_mem_usage":32768,"reinsert_count":0,"update_count":19}
I20260806 07:58:31.759775 7016 ts_tablet_manager.cc:1950] T f6c454c14d784d5f9acb483fc8d26ae3 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:31.760257 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet a1ce9afe0d394368a97a22242d3e403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.760234 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 5eef6e56fda745a3aa131e3db6a0d13d (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.760448 6755 tablet_replica.cc:333] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.760586 6017 catalog_manager.cc:5984] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Sending DeleteTablet for 3 replicas of tablet 85cd55230a604f4c8c22594e2df60249
I20260806 07:58:31.760752 6755 raft_consensus.cc:2243] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.761243 7016 tablet_replica.cc:333] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:31.761283 6148 tablet_service.cc:1558] Processing DeleteTablet for tablet 85cd55230a604f4c8c22594e2df60249 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:39230
I20260806 07:58:31.761353 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet f6c454c14d784d5f9acb483fc8d26ae3 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.761461 7016 raft_consensus.cc:2243] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.761646 7017 ts_tablet_manager.cc:1929] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.761649 6755 raft_consensus.cc:2272] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.761703 7016 raft_consensus.cc:2272] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.761919 6282 tablet_service.cc:1558] Processing DeleteTablet for tablet 85cd55230a604f4c8c22594e2df60249 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:36458
I20260806 07:58:31.762758 6422 tablet_service.cc:1558] Processing DeleteTablet for tablet 85cd55230a604f4c8c22594e2df60249 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 07:58:31 UTC) from {username='slave'} at 127.0.0.1:59016
I20260806 07:58:31.762765 7016 ts_tablet_manager.cc:1916] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.762933 6755 ts_tablet_manager.cc:1916] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.763150 7017 log.cc:1199] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/5e6d749166f540a0a83818c2ef5a00e5
I20260806 07:58:31.763830 7017 ts_tablet_manager.cc:1950] T 5e6d749166f540a0a83818c2ef5a00e5 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:31.763897 6017 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46090:
name: "table-4"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "col"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 16
seed: 0
}
}
I20260806 07:58:31.765130 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 5e6d749166f540a0a83818c2ef5a00e5 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
W20260806 07:58:31.765198 6017 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table-4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 07:58:31.765414 7017 tablet_replica.cc:333] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:31.765762 7017 raft_consensus.cc:2243] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:31.766556 7017 raft_consensus.cc:2272] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.767685 7017 ts_tablet_manager.cc:1916] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 07:58:31.771601 6113 consensus_peers.cc:597] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:31.772606 6755 ts_tablet_manager.cc:1929] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.772812 6755 log.cc:1199] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/d1a3a6825393442a8fa5c96a230afa26
I20260806 07:58:31.772965 7016 ts_tablet_manager.cc:1929] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.773144 7016 log.cc:1199] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/5eef6e56fda745a3aa131e3db6a0d13d
I20260806 07:58:31.773567 6755 ts_tablet_manager.cc:1950] T d1a3a6825393442a8fa5c96a230afa26 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.773881 7016 ts_tablet_manager.cc:1950] T 5eef6e56fda745a3aa131e3db6a0d13d P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:31.774732 6755 tablet_replica.cc:333] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.774737 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet d1a3a6825393442a8fa5c96a230afa26 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.774961 7016 tablet_replica.cc:333] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:31.775059 6755 raft_consensus.cc:2243] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.775173 7016 raft_consensus.cc:2243] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.775306 6755 raft_consensus.cc:2272] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.775353 7016 raft_consensus.cc:2272] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.776342 7016 ts_tablet_manager.cc:1916] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.776506 6755 ts_tablet_manager.cc:1916] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.777127 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 5eef6e56fda745a3aa131e3db6a0d13d (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.781018 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.783795 7016 ts_tablet_manager.cc:1929] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.783938 7016 log.cc:1199] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/d1a3a6825393442a8fa5c96a230afa26
I20260806 07:58:31.784389 7016 ts_tablet_manager.cc:1950] T d1a3a6825393442a8fa5c96a230afa26 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:31.785637 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet d1a3a6825393442a8fa5c96a230afa26 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.785725 7016 tablet_replica.cc:333] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:31.785923 7016 raft_consensus.cc:2243] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 07:58:31.785972 5999 catalog_manager.cc:4995] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): delete failed for tablet f6c454c14d784d5f9acb483fc8d26ae3 because the tablet was not found. No further retry: Not found: Tablet not found: f6c454c14d784d5f9acb483fc8d26ae3
I20260806 07:58:31.786242 7016 raft_consensus.cc:2272] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.787295 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.788204 7017 ts_tablet_manager.cc:1929] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.788516 7017 log.cc:1199] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/5eef6e56fda745a3aa131e3db6a0d13d
I20260806 07:58:31.789093 7017 ts_tablet_manager.cc:1950] T 5eef6e56fda745a3aa131e3db6a0d13d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:31.790696 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 5eef6e56fda745a3aa131e3db6a0d13d (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.791085 7017 tablet_replica.cc:333] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:31.791355 7017 raft_consensus.cc:2243] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:31.791904 7017 raft_consensus.cc:2272] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.793118 6947 consensus_queue.cc:1048] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.793470 7016 ts_tablet_manager.cc:1916] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.795145 6755 ts_tablet_manager.cc:1929] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.795331 6755 log.cc:1199] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/f6c454c14d784d5f9acb483fc8d26ae3
I20260806 07:58:31.795862 6755 ts_tablet_manager.cc:1950] T f6c454c14d784d5f9acb483fc8d26ae3 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.797119 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet f6c454c14d784d5f9acb483fc8d26ae3 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.801412 6568 consensus_queue.cc:1048] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 07:58:31.804781 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.023s user 0.012s sys 0.002s Metrics: {"bytes_written":6942,"cfile_init":1,"dirs.queue_time_us":50,"dirs.run_cpu_time_us":141,"dirs.run_wall_time_us":901,"drs_written":1,"lbm_read_time_us":113,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1050,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":17}
I20260806 07:58:31.805382 6755 tablet_replica.cc:333] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.805693 6755 raft_consensus.cc:2243] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.806102 6755 raft_consensus.cc:2272] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.810359 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:31.811278 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:31.811761 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:31.811932 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260806 07:58:31.812543 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:31.813076 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.813412 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.814078 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.376066
I20260806 07:58:31.818917 7016 ts_tablet_manager.cc:1929] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.819140 7016 log.cc:1199] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/81e7cd5a08d94c519285f18f66c01f0d
I20260806 07:58:31.819893 6755 ts_tablet_manager.cc:1916] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.822264 6148 tablet_service.cc:1511] Processing CreateTablet for tablet fa0d0c42fb3e4fdb9a052d6c21075990 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.822791 6148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0d0c42fb3e4fdb9a052d6c21075990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.823493 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.376066
I20260806 07:58:31.824430 6282 tablet_service.cc:1511] Processing CreateTablet for tablet fa0d0c42fb3e4fdb9a052d6c21075990 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.825101 6282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0d0c42fb3e4fdb9a052d6c21075990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.825791 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.013s user 0.007s sys 0.000s Metrics: {"bytes_written":687,"delete_count":0,"lbm_write_time_us":331,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":16}
I20260806 07:58:31.828233 6289 tablet_service.cc:1511] Processing CreateTablet for tablet acbafefba19f454e8e15f72bb06f8c6f (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.828346 6155 tablet_service.cc:1511] Processing CreateTablet for tablet acbafefba19f454e8e15f72bb06f8c6f (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.828651 6289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acbafefba19f454e8e15f72bb06f8c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.829499 6145 tablet_service.cc:1511] Processing CreateTablet for tablet 6828e964317544099e1886f4398a0322 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.830973 6422 tablet_service.cc:1511] Processing CreateTablet for tablet fa0d0c42fb3e4fdb9a052d6c21075990 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.831430 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0d0c42fb3e4fdb9a052d6c21075990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.831478 6159 tablet_service.cc:1511] Processing CreateTablet for tablet f0e900ebca2c44f381f01492b0601036 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.831573 6420 tablet_service.cc:1511] Processing CreateTablet for tablet acbafefba19f454e8e15f72bb06f8c6f (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.832115 6420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acbafefba19f454e8e15f72bb06f8c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.832414 6156 tablet_service.cc:1511] Processing CreateTablet for tablet 2c7262a4ea974a96be2234607b60b179 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.832850 6155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acbafefba19f454e8e15f72bb06f8c6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.833386 6159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0e900ebca2c44f381f01492b0601036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.833869 6156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7262a4ea974a96be2234607b60b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.834506 6145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6828e964317544099e1886f4398a0322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.834738 6419 tablet_service.cc:1511] Processing CreateTablet for tablet 6828e964317544099e1886f4398a0322 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.835198 6419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6828e964317544099e1886f4398a0322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.836583 6755 ts_tablet_manager.cc:1929] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.836798 6755 log.cc:1199] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/81e7cd5a08d94c519285f18f66c01f0d
I20260806 07:58:31.837487 6755 ts_tablet_manager.cc:1950] T 81e7cd5a08d94c519285f18f66c01f0d P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:31.838907 6755 tablet_replica.cc:333] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:31.839166 6755 raft_consensus.cc:2243] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:31.839121 6290 tablet_service.cc:1511] Processing CreateTablet for tablet 6828e964317544099e1886f4398a0322 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.839527 6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6828e964317544099e1886f4398a0322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.839566 6755 raft_consensus.cc:2272] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.842034 6296 tablet_service.cc:1511] Processing CreateTablet for tablet f0e900ebca2c44f381f01492b0601036 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.842648 6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0e900ebca2c44f381f01492b0601036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.846298 7017 ts_tablet_manager.cc:1916] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.847023 6288 tablet_service.cc:1511] Processing CreateTablet for tablet 2c7262a4ea974a96be2234607b60b179 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.847293 6153 tablet_service.cc:1511] Processing CreateTablet for tablet f4d84432a5b54420ab064ff63612b8bc (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.847452 6288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7262a4ea974a96be2234607b60b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.819777 7016 ts_tablet_manager.cc:1950] T 81e7cd5a08d94c519285f18f66c01f0d P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:31.849798 7016 tablet_replica.cc:333] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:31.850055 7016 raft_consensus.cc:2243] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:31.850241 7016 raft_consensus.cc:2272] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:31.850997 7020 tablet_bootstrap.cc:492] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:31.851250 6281 tablet_service.cc:1511] Processing CreateTablet for tablet e734f1ada5024c07b124303637baadab (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.853886 6429 tablet_service.cc:1511] Processing CreateTablet for tablet e734f1ada5024c07b124303637baadab (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.854348 6429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e734f1ada5024c07b124303637baadab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.854475 6418 tablet_service.cc:1511] Processing CreateTablet for tablet 2c7262a4ea974a96be2234607b60b179 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.854496 7020 tablet_bootstrap.cc:654] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:31.857460 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 81e7cd5a08d94c519285f18f66c01f0d (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.857892 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 81e7cd5a08d94c519285f18f66c01f0d (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:31.859643 6755 ts_tablet_manager.cc:1916] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.860612 6286 tablet_service.cc:1511] Processing CreateTablet for tablet 50c57f5c236c49f1a2cffa660a9854da (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.861639 7016 ts_tablet_manager.cc:1916] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:31.861862 6418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7262a4ea974a96be2234607b60b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.862685 6422 tablet_service.cc:1511] Processing CreateTablet for tablet f0e900ebca2c44f381f01492b0601036 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.862830 5928 tablet.cc:2106] T 00000000000000000000000000000000 P a3d2d029e02f42bdb166bf54079a7128: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:31.863189 6291 tablet_service.cc:1511] Processing CreateTablet for tablet 0c515ba9095b4a13bc21eea1bed0ae7a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.863843 6426 tablet_service.cc:1511] Processing CreateTablet for tablet 50c57f5c236c49f1a2cffa660a9854da (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.864295 6432 tablet_service.cc:1511] Processing CreateTablet for tablet 9b62ecd72f514f308cf3dd80a25d916d (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.864598 6433 tablet_service.cc:1511] Processing CreateTablet for tablet f4d84432a5b54420ab064ff63612b8bc (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.867209 6432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b62ecd72f514f308cf3dd80a25d916d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.867904 6426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50c57f5c236c49f1a2cffa660a9854da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.868144 6146 tablet_service.cc:1511] Processing CreateTablet for tablet 50c57f5c236c49f1a2cffa660a9854da (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.867219 6428 tablet_service.cc:1511] Processing CreateTablet for tablet c0a3d34e6ca143c7834aaa4a9a17b2c3 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.869194 6150 tablet_service.cc:1511] Processing CreateTablet for tablet e734f1ada5024c07b124303637baadab (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.869542 6153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d84432a5b54420ab064ff63612b8bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.870081 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.041s user 0.010s sys 0.000s Metrics: {"dirs.queue_time_us":3040,"dirs.run_cpu_time_us":725,"dirs.run_wall_time_us":3353,"lbm_write_time_us":1769,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":6,"mutex_wait_us":2341,"num_input_rowsets":0,"spinlock_wait_cycles":1280,"thread_start_us":5595,"threads_started":1}
I20260806 07:58:31.870435 6284 tablet_service.cc:1511] Processing CreateTablet for tablet 4e954825b5e44e4ebd01e742b8ff0bf9 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.870688 6295 tablet_service.cc:1511] Processing CreateTablet for tablet c0a3d34e6ca143c7834aaa4a9a17b2c3 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.870865 6283 tablet_service.cc:1511] Processing CreateTablet for tablet 9b62ecd72f514f308cf3dd80a25d916d (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.871240 6287 tablet_service.cc:1511] Processing CreateTablet for tablet f4d84432a5b54420ab064ff63612b8bc (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.871618 6158 tablet_service.cc:1511] Processing CreateTablet for tablet c0a3d34e6ca143c7834aaa4a9a17b2c3 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 9, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.872136 6158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a3d34e6ca143c7834aaa4a9a17b2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.873654 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:31.874070 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:31.874490 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 249 bytes on disk
I20260806 07:58:31.875180 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4}
I20260806 07:58:31.875572 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.876150 6294 tablet_service.cc:1511] Processing CreateTablet for tablet 61f1edf7136a409bb7e4232596b305b0 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.876354 6285 tablet_service.cc:1511] Processing CreateTablet for tablet 8af6c9fa17d04ed6b54fa9e8b190bc63 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.877025 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.877396 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.079s user 0.018s sys 0.001s Metrics: {"bytes_written":687,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":11715,"dirs.run_cpu_time_us":182,"dirs.run_wall_time_us":1312,"drs_written":1,"lbm_read_time_us":98,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1292,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"reinsert_count":0,"rows_written":17,"update_count":16}
I20260806 07:58:31.877689 6293 tablet_service.cc:1511] Processing CreateTablet for tablet f89cbb223d2446f789aa88d6a6ec778a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.879750 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.930331
I20260806 07:58:31.880511 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.930331
I20260806 07:58:31.881131 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling CompactRowSetsOp(00000000000000000000000000000000): perf score=0.930331
I20260806 07:58:31.882234 6433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d84432a5b54420ab064ff63612b8bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.886730 6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a3d34e6ca143c7834aaa4a9a17b2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.887810 6424 tablet_service.cc:1511] Processing CreateTablet for tablet 4e954825b5e44e4ebd01e742b8ff0bf9 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.888821 6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e954825b5e44e4ebd01e742b8ff0bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.890331 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.344706
I20260806 07:58:31.890592 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.891292 6425 tablet_service.cc:1511] Processing CreateTablet for tablet 0c515ba9095b4a13bc21eea1bed0ae7a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.892170 6430 tablet_service.cc:1511] Processing CreateTablet for tablet 61f1edf7136a409bb7e4232596b305b0 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.894381 6151 tablet_service.cc:1511] Processing CreateTablet for tablet 9b62ecd72f514f308cf3dd80a25d916d (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 8, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.895097 6154 tablet_service.cc:1511] Processing CreateTablet for tablet 4e954825b5e44e4ebd01e742b8ff0bf9 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 10, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.895653 6150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e734f1ada5024c07b124303637baadab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.898666 6281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e734f1ada5024c07b124303637baadab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.898802 6147 tablet_service.cc:1511] Processing CreateTablet for tablet 0c515ba9095b4a13bc21eea1bed0ae7a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 11, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.899325 6157 tablet_service.cc:1511] Processing CreateTablet for tablet 61f1edf7136a409bb7e4232596b305b0 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 12, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.899900 6292 tablet_service.cc:1511] Processing CreateTablet for tablet 1f789d0955264a6faefbd09df6964b5a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.900215 6291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c515ba9095b4a13bc21eea1bed0ae7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.900490 6286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50c57f5c236c49f1a2cffa660a9854da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.901095 6284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e954825b5e44e4ebd01e742b8ff0bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.901665 6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89cbb223d2446f789aa88d6a6ec778a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.902323 6283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b62ecd72f514f308cf3dd80a25d916d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.903386 6149 tablet_service.cc:1511] Processing CreateTablet for tablet 8af6c9fa17d04ed6b54fa9e8b190bc63 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.903816 6151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b62ecd72f514f308cf3dd80a25d916d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.904230 6152 tablet_service.cc:1511] Processing CreateTablet for tablet f89cbb223d2446f789aa88d6a6ec778a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.905797 6285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af6c9fa17d04ed6b54fa9e8b190bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.906318 6160 tablet_service.cc:1511] Processing CreateTablet for tablet 1f789d0955264a6faefbd09df6964b5a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.909562 6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0e900ebca2c44f381f01492b0601036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.911417 6423 tablet_service.cc:1511] Processing CreateTablet for tablet f89cbb223d2446f789aa88d6a6ec778a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 13, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.918267 6287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d84432a5b54420ab064ff63612b8bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.918628 6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f1edf7136a409bb7e4232596b305b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.919802 6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a3d34e6ca143c7834aaa4a9a17b2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.920243 6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f789d0955264a6faefbd09df6964b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.923326 6425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c515ba9095b4a13bc21eea1bed0ae7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.924144 6421 tablet_service.cc:1511] Processing CreateTablet for tablet 1f789d0955264a6faefbd09df6964b5a (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 15, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:31.924674 6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89cbb223d2446f789aa88d6a6ec778a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.936664 6149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af6c9fa17d04ed6b54fa9e8b190bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.957034 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5347 bytes on disk
I20260806 07:58:31.958184 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":118,"lbm_reads_lt_1ms":4}
I20260806 07:58:31.959827 6154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e954825b5e44e4ebd01e742b8ff0bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.965925 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5347 bytes on disk
I20260806 07:58:31.966449 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:31.975852 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.161s user 0.033s sys 0.016s Metrics: {"bytes_written":38921,"cfile_cache_hit":14,"cfile_cache_hit_bytes":27773,"cfile_cache_miss":38,"cfile_cache_miss_bytes":40793,"cfile_init":10,"delta_iterators_relevant":9,"dirs.queue_time_us":3941,"dirs.run_cpu_time_us":1298,"dirs.run_wall_time_us":9774,"drs_written":1,"lbm_read_time_us":1726,"lbm_reads_lt_1ms":78,"lbm_write_time_us":3065,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":154601,"rows_written":88,"thread_start_us":20333,"threads_started":2}
I20260806 07:58:31.977666 6146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50c57f5c236c49f1a2cffa660a9854da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.985361 6147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c515ba9095b4a13bc21eea1bed0ae7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.995412 6755 ts_tablet_manager.cc:1929] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:31.995723 6157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f1edf7136a409bb7e4232596b305b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:31.996230 6160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f789d0955264a6faefbd09df6964b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:32.009173 7022 tablet_bootstrap.cc:492] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.010600 6152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89cbb223d2446f789aa88d6a6ec778a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260806 07:58:32.014683 6113 consensus_peers.cc:597] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81e7cd5a08d94c519285f18f66c01f0d. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.032052 6755 log.cc:1199] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/7f80b3d857bd4e62b71bdd63157e8872
I20260806 07:58:32.032701 6755 ts_tablet_manager.cc:1950] T 7f80b3d857bd4e62b71bdd63157e8872 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.034641 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 7f80b3d857bd4e62b71bdd63157e8872 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.040944 6419 tablet_service.cc:1511] Processing CreateTablet for tablet 8af6c9fa17d04ed6b54fa9e8b190bc63 (DEFAULT_TABLE table=table-4 [id=a63fabacd6eb464b85e53db3d6ab846c]), partition=HASH (key) PARTITION 14, RANGE (key) PARTITION UNBOUNDED
I20260806 07:58:32.047927 6421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f789d0955264a6faefbd09df6964b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:32.053647 7022 tablet_bootstrap.cc:654] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.063186 6755 tablet_replica.cc:333] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.064294 6755 raft_consensus.cc:2243] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 07:58:32.065387 6113 consensus_peers.cc:597] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.066047 6755 raft_consensus.cc:2272] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.071841 6755 ts_tablet_manager.cc:1916] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.078261 6430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f1edf7136a409bb7e4232596b305b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:32.078616 6419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af6c9fa17d04ed6b54fa9e8b190bc63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 07:58:32.089959 6755 ts_tablet_manager.cc:1929] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.090161 6755 log.cc:1199] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/d2582adff5654c49b98f072ddfa3ff49
I20260806 07:58:32.090814 6755 ts_tablet_manager.cc:1950] T d2582adff5654c49b98f072ddfa3ff49 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.092326 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet d2582adff5654c49b98f072ddfa3ff49 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.092935 6755 tablet_replica.cc:333] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.093302 6755 raft_consensus.cc:2243] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.093873 6755 raft_consensus.cc:2272] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.095160 6755 ts_tablet_manager.cc:1916] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.103806 6755 ts_tablet_manager.cc:1929] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.104043 6755 log.cc:1199] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/2746179ffc5641c393439e5292ac0b69
I20260806 07:58:32.104578 6755 ts_tablet_manager.cc:1950] T 2746179ffc5641c393439e5292ac0b69 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.106015 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 2746179ffc5641c393439e5292ac0b69 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.106603 6755 tablet_replica.cc:333] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.106933 6755 raft_consensus.cc:2243] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.110916 6755 raft_consensus.cc:2272] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.112088 6755 ts_tablet_manager.cc:1916] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.121645 6755 ts_tablet_manager.cc:1929] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.121970 6755 log.cc:1199] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/b13ae96de4eb42088b187c4efc95802a
I20260806 07:58:32.122618 6755 ts_tablet_manager.cc:1950] T b13ae96de4eb42088b187c4efc95802a P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.124220 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet b13ae96de4eb42088b187c4efc95802a (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.124895 6755 tablet_replica.cc:333] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.125303 6755 raft_consensus.cc:2243] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.125633 6755 raft_consensus.cc:2272] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.126899 6755 ts_tablet_manager.cc:1916] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.130455 7024 tablet_bootstrap.cc:492] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:32.132015 6002 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
W20260806 07:58:32.137703 6114 consensus_peers.cc:597] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Couldn't send request to peer 5d4b117032604585af3544cb030a10f2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81e7cd5a08d94c519285f18f66c01f0d. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.134832 6755 ts_tablet_manager.cc:1929] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.143558 6755 log.cc:1199] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/5a6182cc1daf4f6ca2c2464b6977cde1
I20260806 07:58:32.144070 6755 ts_tablet_manager.cc:1950] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.146497 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 5a6182cc1daf4f6ca2c2464b6977cde1 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.146678 6755 tablet_replica.cc:333] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.146942 6755 raft_consensus.cc:2243] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 2 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.147572 6755 raft_consensus.cc:2272] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.149787 6755 ts_tablet_manager.cc:1916] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.151372 7024 tablet_bootstrap.cc:654] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.161751 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.263s user 0.006s sys 0.007s Metrics: {"dirs.queue_time_us":4175,"dirs.run_cpu_time_us":1014,"dirs.run_wall_time_us":11861,"lbm_write_time_us":496,"lbm_writes_lt_1ms":7,"num_input_rowsets":0,"thread_start_us":240481,"threads_started":2}
I20260806 07:58:32.164626 6004 tablet.cc:2106] T 00000000000000000000000000000000 P 15b584d20bab4416a9d975df6675b207: Compaction resulted in no output rows (all input rows were GCed!) Removing all input rowsets.
I20260806 07:58:32.160234 6755 ts_tablet_manager.cc:1929] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260806 07:58:32.164961 6755 log.cc:1199] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/be96d575a672438994b6c6364180ec84
I20260806 07:58:32.165647 6755 ts_tablet_manager.cc:1950] T be96d575a672438994b6c6364180ec84 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.166965 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet be96d575a672438994b6c6364180ec84 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.178692 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.299s user 0.012s sys 0.000s Metrics: {"dirs.queue_time_us":14234,"dirs.run_cpu_time_us":1019,"dirs.run_wall_time_us":5518,"lbm_write_time_us":578,"lbm_writes_lt_1ms":7,"mutex_wait_us":12382,"num_input_rowsets":0}
I20260806 07:58:32.180066 7020 tablet_bootstrap.cc:492] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.180352 7020 ts_tablet_manager.cc:1403] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.329s user 0.004s sys 0.002s
I20260806 07:58:32.184204 7017 ts_tablet_manager.cc:1929] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.184361 7017 log.cc:1199] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/d1a3a6825393442a8fa5c96a230afa26
I20260806 07:58:32.184880 7017 ts_tablet_manager.cc:1950] T d1a3a6825393442a8fa5c96a230afa26 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.185971 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet d1a3a6825393442a8fa5c96a230afa26 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.194087 7017 tablet_replica.cc:333] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.194393 7017 raft_consensus.cc:2243] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.194834 7017 raft_consensus.cc:2272] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.195835 7017 ts_tablet_manager.cc:1916] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.196276 7020 raft_consensus.cc:359] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.196521 7020 raft_consensus.cc:385] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.196620 7020 raft_consensus.cc:740] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.196868 7020 consensus_queue.cc:260] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.197697 7020 ts_tablet_manager.cc:1434] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260806 07:58:32.198173 7020 tablet_bootstrap.cc:492] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.207448 7022 tablet_bootstrap.cc:492] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.208051 7022 ts_tablet_manager.cc:1403] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.237s user 0.003s sys 0.004s
I20260806 07:58:32.209744 7022 raft_consensus.cc:359] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.210373 7022 raft_consensus.cc:385] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.210532 7022 raft_consensus.cc:740] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.210923 7022 consensus_queue.cc:260] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.213059 7022 ts_tablet_manager.cc:1434] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260806 07:58:32.213587 7022 tablet_bootstrap.cc:492] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.216985 7016 ts_tablet_manager.cc:1929] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.217144 7016 log.cc:1199] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/7f80b3d857bd4e62b71bdd63157e8872
I20260806 07:58:32.217202 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5347 bytes on disk
I20260806 07:58:32.217562 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5347 bytes on disk
I20260806 07:58:32.217641 7016 ts_tablet_manager.cc:1950] T 7f80b3d857bd4e62b71bdd63157e8872 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.217926 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 5347 bytes on disk
I20260806 07:58:32.218678 7016 tablet_replica.cc:333] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.218643 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":112,"lbm_reads_lt_1ms":4}
I20260806 07:58:32.218809 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 7f80b3d857bd4e62b71bdd63157e8872 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.218886 7016 raft_consensus.cc:2243] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.219098 7016 raft_consensus.cc:2272] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.219136 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.219507 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.220134 7020 tablet_bootstrap.cc:654] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.167272 6755 tablet_replica.cc:333] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.178624 7024 tablet_bootstrap.cc:492] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:32.221696 6755 raft_consensus.cc:2243] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.221885 7024 ts_tablet_manager.cc:1403] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.092s user 0.003s sys 0.003s
I20260806 07:58:32.221902 7022 tablet_bootstrap.cc:654] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.221916 6755 raft_consensus.cc:2272] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.222503 7020 tablet_bootstrap.cc:492] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.222651 7020 ts_tablet_manager.cc:1403] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260806 07:58:32.222831 6755 ts_tablet_manager.cc:1916] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.222921 7024 raft_consensus.cc:359] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.223212 7024 raft_consensus.cc:385] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.223364 7024 raft_consensus.cc:740] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.223652 7024 consensus_queue.cc:260] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.224300 7016 ts_tablet_manager.cc:1916] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.224570 7024 ts_tablet_manager.cc:1434] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 07:58:32.227730 7024 tablet_bootstrap.cc:492] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:32.227778 7020 raft_consensus.cc:359] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.228070 7020 raft_consensus.cc:385] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.228155 7020 raft_consensus.cc:740] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.229923 7020 consensus_queue.cc:260] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.230440 7020 ts_tablet_manager.cc:1434] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 07:58:32.230849 7020 tablet_bootstrap.cc:492] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.238940 7017 ts_tablet_manager.cc:1929] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.239095 6755 ts_tablet_manager.cc:1929] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.239148 7017 log.cc:1199] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/f6c454c14d784d5f9acb483fc8d26ae3
I20260806 07:58:32.239327 6755 log.cc:1199] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/dbc52165692848699c405d3e54739402
I20260806 07:58:32.239843 6755 ts_tablet_manager.cc:1950] T dbc52165692848699c405d3e54739402 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.240338 7017 ts_tablet_manager.cc:1950] T f6c454c14d784d5f9acb483fc8d26ae3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.241022 6755 tablet_replica.cc:333] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.241573 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet f6c454c14d784d5f9acb483fc8d26ae3 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.241781 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet dbc52165692848699c405d3e54739402 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.242077 7017 tablet_replica.cc:333] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.242384 7017 raft_consensus.cc:2243] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.242830 7017 raft_consensus.cc:2272] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.243112 7022 tablet_bootstrap.cc:492] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.243273 7022 ts_tablet_manager.cc:1403] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260806 07:58:32.243443 7017 ts_tablet_manager.cc:1916] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.244024 7024 tablet_bootstrap.cc:654] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.245141 7022 raft_consensus.cc:359] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.245390 7022 raft_consensus.cc:385] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.245499 7022 raft_consensus.cc:740] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.245854 7022 consensus_queue.cc:260] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.247879 7022 ts_tablet_manager.cc:1434] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 07:58:32.249019 7022 tablet_bootstrap.cc:492] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.253443 7022 tablet_bootstrap.cc:654] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.254583 6755 raft_consensus.cc:2243] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.255699 6755 raft_consensus.cc:2272] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.256858 6755 ts_tablet_manager.cc:1916] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.258659 7024 tablet_bootstrap.cc:492] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:32.259032 7024 ts_tablet_manager.cc:1403] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.008s
I20260806 07:58:32.260257 7024 raft_consensus.cc:359] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.260598 7024 raft_consensus.cc:385] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.260752 7024 raft_consensus.cc:740] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.261169 7024 consensus_queue.cc:260] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.262144 7020 tablet_bootstrap.cc:654] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.268150 6885 raft_consensus.cc:493] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.268438 6885 raft_consensus.cc:515] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.268605 7016 ts_tablet_manager.cc:1929] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.268731 7016 log.cc:1199] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/d2582adff5654c49b98f072ddfa3ff49
I20260806 07:58:32.269459 6885 leader_election.cc:290] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.270298 7024 ts_tablet_manager.cc:1434] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260806 07:58:32.272497 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0d0c42fb3e4fdb9a052d6c21075990" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.272707 6452 raft_consensus.cc:2468] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:32.273697 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0d0c42fb3e4fdb9a052d6c21075990" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.273988 6312 raft_consensus.cc:2468] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:32.275179 6114 leader_election.cc:304] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:32.276049 6885 raft_consensus.cc:2804] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.276170 6885 raft_consensus.cc:493] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.276175 7024 tablet_bootstrap.cc:492] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:32.276264 6885 raft_consensus.cc:3060] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.279248 7017 ts_tablet_manager.cc:1929] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.278899 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: CompactRowSetsOp(00000000000000000000000000000000) complete. Timing: real 0.397s user 0.061s sys 0.039s Metrics: {"bytes_written":38921,"cfile_cache_hit":30,"cfile_cache_hit_bytes":28228,"cfile_cache_miss":78,"cfile_cache_miss_bytes":46098,"cfile_init":22,"delta_iterators_relevant":30,"dirs.queue_time_us":38479,"dirs.run_cpu_time_us":1786,"dirs.run_wall_time_us":24242,"drs_written":1,"lbm_read_time_us":14105,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":164,"lbm_write_time_us":1716,"lbm_writes_lt_1ms":28,"mutex_wait_us":4152,"num_input_rowsets":3,"peak_mem_usage":173055,"rows_written":88,"thread_start_us":57424,"threads_started":2}
I20260806 07:58:32.279716 6885 raft_consensus.cc:515] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.280407 7016 ts_tablet_manager.cc:1950] T d2582adff5654c49b98f072ddfa3ff49 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.281101 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0d0c42fb3e4fdb9a052d6c21075990" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:32.281329 7024 tablet_bootstrap.cc:654] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.281385 6312 raft_consensus.cc:3060] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.282065 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet d2582adff5654c49b98f072ddfa3ff49 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.282713 7022 tablet_bootstrap.cc:492] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.282958 7022 ts_tablet_manager.cc:1403] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.000s
I20260806 07:58:32.284093 7022 raft_consensus.cc:359] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.284380 7022 raft_consensus.cc:385] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.284503 7022 raft_consensus.cc:740] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.284974 7016 tablet_replica.cc:333] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.284865 7022 consensus_queue.cc:260] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.285336 7016 raft_consensus.cc:2243] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.285550 7016 raft_consensus.cc:2272] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.285871 7022 ts_tablet_manager.cc:1434] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 07:58:32.286324 7016 ts_tablet_manager.cc:1916] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.286453 7022 tablet_bootstrap.cc:492] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.288190 6312 raft_consensus.cc:2468] T fa0d0c42fb3e4fdb9a052d6c21075990 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:32.288887 6113 leader_election.cc:304] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:32.289237 6544 raft_consensus.cc:2804] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:32.279860 7017 log.cc:1199] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/81e7cd5a08d94c519285f18f66c01f0d
I20260806 07:58:32.291126 6544 raft_consensus.cc:697] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:32.291167 6755 ts_tablet_manager.cc:1929] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.291445 6755 log.cc:1199] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/c31bd22c7f6e47c595747dec77a8cc70
I20260806 07:58:32.291524 7020 tablet_bootstrap.cc:492] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.291685 7020 ts_tablet_manager.cc:1403] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.000s
I20260806 07:58:32.292189 6755 ts_tablet_manager.cc:1950] T c31bd22c7f6e47c595747dec77a8cc70 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.292652 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0d0c42fb3e4fdb9a052d6c21075990" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:32.292876 6452 raft_consensus.cc:3060] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.293027 7020 raft_consensus.cc:359] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.293237 7020 raft_consensus.cc:385] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.293349 7020 raft_consensus.cc:740] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.293648 6755 tablet_replica.cc:333] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.293720 7020 consensus_queue.cc:260] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.294034 6755 raft_consensus.cc:2243] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.294281 6755 raft_consensus.cc:2272] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.294426 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet c31bd22c7f6e47c595747dec77a8cc70 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.294971 7020 ts_tablet_manager.cc:1434] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 07:58:32.295356 7020 tablet_bootstrap.cc:492] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.295341 6544 consensus_queue.cc:237] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.295711 6755 ts_tablet_manager.cc:1916] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.296658 6885 leader_election.cc:290] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.296888 7017 ts_tablet_manager.cc:1950] T 81e7cd5a08d94c519285f18f66c01f0d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.298293 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 81e7cd5a08d94c519285f18f66c01f0d (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.300671 7017 tablet_replica.cc:333] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.301112 7017 raft_consensus.cc:2243] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.301434 7017 raft_consensus.cc:2272] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.301908 7022 tablet_bootstrap.cc:654] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.305069 6755 ts_tablet_manager.cc:1929] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.305241 6755 log.cc:1199] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/f9aa031a25104549905f7a7e1007187f
I20260806 07:58:32.305909 6755 ts_tablet_manager.cc:1950] T f9aa031a25104549905f7a7e1007187f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.307070 6755 tablet_replica.cc:333] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.307193 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet f9aa031a25104549905f7a7e1007187f (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.307370 6755 raft_consensus.cc:2243] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.307614 6755 raft_consensus.cc:2272] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.307556 7017 ts_tablet_manager.cc:1916] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 07:58:32.308508 6384 consensus_peers.cc:597] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c31bd22c7f6e47c595747dec77a8cc70. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.308813 6755 ts_tablet_manager.cc:1916] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.311128 6452 raft_consensus.cc:2468] T fa0d0c42fb3e4fdb9a052d6c21075990 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:32.313539 7020 tablet_bootstrap.cc:654] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.315287 7024 tablet_bootstrap.cc:492] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:32.315521 7024 ts_tablet_manager.cc:1403] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.017s
I20260806 07:58:32.316644 7024 raft_consensus.cc:359] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.317162 7024 raft_consensus.cc:385] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.317472 7024 raft_consensus.cc:740] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.318029 7024 consensus_queue.cc:260] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.319514 6755 ts_tablet_manager.cc:1929] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.319641 6755 log.cc:1199] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/a1ce9afe0d394368a97a22242d3e403f
I20260806 07:58:32.320204 6755 ts_tablet_manager.cc:1950] T a1ce9afe0d394368a97a22242d3e403f P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.321354 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet a1ce9afe0d394368a97a22242d3e403f (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.321979 6755 tablet_replica.cc:333] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: stopping tablet replica
I20260806 07:58:32.322245 6755 raft_consensus.cc:2243] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.322685 6755 raft_consensus.cc:2272] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.324465 6755 ts_tablet_manager.cc:1916] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.325027 6953 raft_consensus.cc:493] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.325318 6953 raft_consensus.cc:515] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.326275 6953 leader_election.cc:290] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:32.327814 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6828e964317544099e1886f4398a0322" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.327903 7024 ts_tablet_manager.cc:1434] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.012s user 0.001s sys 0.002s
I20260806 07:58:32.328581 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6828e964317544099e1886f4398a0322" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:32.328838 7024 tablet_bootstrap.cc:492] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
W20260806 07:58:32.329069 6384 leader_election.cc:343] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.329874 6178 raft_consensus.cc:2468] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:32.330943 6387 leader_election.cc:304] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:32.332229 7024 tablet_bootstrap.cc:654] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.333348 6017 catalog_manager.cc:5697] T fa0d0c42fb3e4fdb9a052d6c21075990 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:32.335232 6953 raft_consensus.cc:2804] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.335381 6953 raft_consensus.cc:493] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.335546 6953 raft_consensus.cc:3060] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.336737 7017 ts_tablet_manager.cc:1929] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.336915 7017 log.cc:1199] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/7f80b3d857bd4e62b71bdd63157e8872
I20260806 07:58:32.338792 7022 tablet_bootstrap.cc:492] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.338996 7022 ts_tablet_manager.cc:1403] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.004s
I20260806 07:58:32.339811 7022 raft_consensus.cc:359] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.340022 7022 raft_consensus.cc:385] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.340176 7022 raft_consensus.cc:740] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.340579 7022 consensus_queue.cc:260] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.341107 7020 tablet_bootstrap.cc:492] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.341275 7022 ts_tablet_manager.cc:1434] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 07:58:32.341379 7020 ts_tablet_manager.cc:1403] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.006s
I20260806 07:58:32.342866 7022 tablet_bootstrap.cc:492] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.344137 7020 raft_consensus.cc:359] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.344359 7020 raft_consensus.cc:385] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.345348 7022 tablet_bootstrap.cc:654] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.345573 6755 ts_tablet_manager.cc:1929] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.345767 6755 log.cc:1199] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-1/wal/wals/85cd55230a604f4c8c22594e2df60249
I20260806 07:58:32.345254 6953 raft_consensus.cc:515] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.346401 7017 ts_tablet_manager.cc:1950] T 7f80b3d857bd4e62b71bdd63157e8872 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.345348 7020 raft_consensus.cc:740] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.346729 7020 consensus_queue.cc:260] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.347584 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 7f80b3d857bd4e62b71bdd63157e8872 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.347816 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6828e964317544099e1886f4398a0322" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:32.347975 7017 tablet_replica.cc:333] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.348086 6178 raft_consensus.cc:3060] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.348186 7017 raft_consensus.cc:2243] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.348667 7017 raft_consensus.cc:2272] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.349917 7017 ts_tablet_manager.cc:1916] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.351387 7020 ts_tablet_manager.cc:1434] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260806 07:58:32.352169 6178 raft_consensus.cc:2468] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 1.
I20260806 07:58:32.354193 6387 leader_election.cc:304] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:32.354290 7020 tablet_bootstrap.cc:492] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.354561 6623 raft_consensus.cc:2804] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:32.354724 6623 raft_consensus.cc:697] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Running, Role: LEADER
I20260806 07:58:32.354992 6623 consensus_queue.cc:237] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.356274 7016 ts_tablet_manager.cc:1929] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.356420 7016 log.cc:1199] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/2746179ffc5641c393439e5292ac0b69
I20260806 07:58:32.356791 7020 tablet_bootstrap.cc:654] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.356909 7016 ts_tablet_manager.cc:1950] T 2746179ffc5641c393439e5292ac0b69 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.358042 7016 tablet_replica.cc:333] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.358245 7016 raft_consensus.cc:2243] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.358220 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 2746179ffc5641c393439e5292ac0b69 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.358603 7016 raft_consensus.cc:2272] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.358624 6755 ts_tablet_manager.cc:1950] T 85cd55230a604f4c8c22594e2df60249 P 0d9e5266c4e04896a714d37cd67fdcc7: Deleting consensus metadata
I20260806 07:58:32.360852 5998 catalog_manager.cc:5028] TS 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): tablet 85cd55230a604f4c8c22594e2df60249 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.361852 7022 tablet_bootstrap.cc:492] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.362042 7022 ts_tablet_manager.cc:1403] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.004s
I20260806 07:58:32.362049 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6828e964317544099e1886f4398a0322" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
W20260806 07:58:32.362888 6384 leader_election.cc:343] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.362960 7022 raft_consensus.cc:359] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.363155 7022 raft_consensus.cc:385] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.363229 7022 raft_consensus.cc:740] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.363467 7022 consensus_queue.cc:260] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.363955 7022 ts_tablet_manager.cc:1434] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:32.364271 7022 tablet_bootstrap.cc:492] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.365723 6953 leader_election.cc:290] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:32.366358 7022 tablet_bootstrap.cc:654] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.365926 7016 ts_tablet_manager.cc:1916] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.369393 7022 tablet_bootstrap.cc:492] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.369643 7022 ts_tablet_manager.cc:1403] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 07:58:32.370541 7022 raft_consensus.cc:359] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.370801 7022 raft_consensus.cc:385] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.370934 7022 raft_consensus.cc:740] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.371438 7022 consensus_queue.cc:260] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.371933 6918 raft_consensus.cc:493] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.372175 6918 raft_consensus.cc:515] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.372995 6918 leader_election.cc:290] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.373459 7022 ts_tablet_manager.cc:1434] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 07:58:32.373991 7022 tablet_bootstrap.cc:492] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.373937 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b62ecd72f514f308cf3dd80a25d916d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.373909 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b62ecd72f514f308cf3dd80a25d916d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:32.374217 6452 raft_consensus.cc:2468] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 0.
W20260806 07:58:32.374547 6252 leader_election.cc:343] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.374763 6250 leader_election.cc:304] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:32.375408 6918 raft_consensus.cc:2804] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.375742 6918 raft_consensus.cc:493] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.375892 6918 raft_consensus.cc:3060] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.376602 7022 tablet_bootstrap.cc:654] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.381433 7020 tablet_bootstrap.cc:492] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.381420 7017 ts_tablet_manager.cc:1929] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.381623 7020 ts_tablet_manager.cc:1403] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.005s
I20260806 07:58:32.381660 7017 log.cc:1199] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/d2582adff5654c49b98f072ddfa3ff49
I20260806 07:58:32.382365 7017 ts_tablet_manager.cc:1950] T d2582adff5654c49b98f072ddfa3ff49 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.382506 7020 raft_consensus.cc:359] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.382692 7020 raft_consensus.cc:385] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.382894 7020 raft_consensus.cc:740] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.383179 7020 consensus_queue.cc:260] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.383602 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet d2582adff5654c49b98f072ddfa3ff49 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.384598 7017 tablet_replica.cc:333] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.384969 7017 raft_consensus.cc:2243] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.385241 7017 raft_consensus.cc:2272] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.385556 7022 tablet_bootstrap.cc:492] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.385797 7022 ts_tablet_manager.cc:1403] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260806 07:58:32.380901 6918 raft_consensus.cc:515] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.386108 7017 ts_tablet_manager.cc:1916] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.387142 7022 raft_consensus.cc:359] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.387395 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b62ecd72f514f308cf3dd80a25d916d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168"
I20260806 07:58:32.387487 7022 raft_consensus.cc:385] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.387691 7022 raft_consensus.cc:740] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
W20260806 07:58:32.388346 6252 leader_election.cc:343] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.389938 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b62ecd72f514f308cf3dd80a25d916d" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:32.389902 7022 consensus_queue.cc:260] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.390231 6918 leader_election.cc:290] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.391098 7022 ts_tablet_manager.cc:1434] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 07:58:32.392038 7022 tablet_bootstrap.cc:492] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.396971 6452 raft_consensus.cc:3060] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.397399 7022 tablet_bootstrap.cc:654] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
W20260806 07:58:32.398756 6384 consensus_peers.cc:597] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9aa031a25104549905f7a7e1007187f. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.402210 6452 raft_consensus.cc:2468] T 9b62ecd72f514f308cf3dd80a25d916d P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9e5266c4e04896a714d37cd67fdcc7 in term 1.
I20260806 07:58:32.403191 6250 leader_election.cc:304] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2; no voters: 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:32.403923 6918 raft_consensus.cc:2804] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:32.408152 7017 ts_tablet_manager.cc:1929] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.408442 7017 log.cc:1199] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/2746179ffc5641c393439e5292ac0b69
I20260806 07:58:32.409245 7017 ts_tablet_manager.cc:1950] T 2746179ffc5641c393439e5292ac0b69 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
W20260806 07:58:32.410549 6384 consensus_peers.cc:597] T a1ce9afe0d394368a97a22242d3e403f P 5d4b117032604585af3544cb030a10f2 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a1ce9afe0d394368a97a22242d3e403f. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.410470 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 2746179ffc5641c393439e5292ac0b69 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.411998 7020 ts_tablet_manager.cc:1434] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.030s user 0.000s sys 0.014s
I20260806 07:58:32.412355 7020 tablet_bootstrap.cc:492] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.423566 7016 ts_tablet_manager.cc:1929] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.423715 7016 log.cc:1199] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/b13ae96de4eb42088b187c4efc95802a
I20260806 07:58:32.425694 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.427675 7016 ts_tablet_manager.cc:1950] T b13ae96de4eb42088b187c4efc95802a P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.427834 6544 raft_consensus.cc:493] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.428093 6544 raft_consensus.cc:515] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.428510 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.428836 7016 tablet_replica.cc:333] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.428912 6544 leader_election.cc:290] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.428925 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.429149 7016 raft_consensus.cc:2243] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.429452 6918 raft_consensus.cc:697] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Running, Role: LEADER
I20260806 07:58:32.429890 6918 consensus_queue.cc:237] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.430228 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbafefba19f454e8e15f72bb06f8c6f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:32.430923 6113 leader_election.cc:343] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.431331 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet b13ae96de4eb42088b187c4efc95802a (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.434362 7020 tablet_bootstrap.cc:654] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.432246 7016 raft_consensus.cc:2272] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.436702 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.004s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":272,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:32.440440 7022 tablet_bootstrap.cc:492] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.440683 7022 ts_tablet_manager.cc:1403] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.010s
I20260806 07:58:32.442168 7022 raft_consensus.cc:359] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.442410 7022 raft_consensus.cc:385] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.442489 7022 raft_consensus.cc:740] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.442873 7017 tablet_replica.cc:333] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.442844 7022 consensus_queue.cc:260] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.443059 7017 raft_consensus.cc:2243] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.443264 7017 raft_consensus.cc:2272] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.444247 7017 ts_tablet_manager.cc:1916] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.444696 7022 ts_tablet_manager.cc:1434] T 9b62ecd72f514f308cf3dd80a25d916d P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 07:58:32.445088 7022 tablet_bootstrap.cc:492] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.447778 7016 ts_tablet_manager.cc:1916] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.448482 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.448804 7022 tablet_bootstrap.cc:654] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.448860 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.449216 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.449584 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.449738 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.450996 6017 catalog_manager.cc:5697] T 9b62ecd72f514f308cf3dd80a25d916d P 0d9e5266c4e04896a714d37cd67fdcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194). New cstate: current_term: 1 leader_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
W20260806 07:58:32.453572 6113 consensus_peers.cc:597] T 85cd55230a604f4c8c22594e2df60249 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85cd55230a604f4c8c22594e2df60249. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.453627 6014 catalog_manager.cc:5697] T 6828e964317544099e1886f4398a0322 P 5d4b117032604585af3544cb030a10f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4b117032604585af3544cb030a10f2 (127.20.202.195). New cstate: current_term: 1 leader_uuid: "5d4b117032604585af3544cb030a10f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260806 07:58:32.454703 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbafefba19f454e8e15f72bb06f8c6f" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.455271 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.003s sys 0.002s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":250,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
W20260806 07:58:32.455448 6114 leader_election.cc:343] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.455585 6114 leader_election.cc:304] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:32.456159 6544 raft_consensus.cc:2749] T acbafefba19f454e8e15f72bb06f8c6f P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 07:58:32.458504 6113 consensus_peers.cc:597] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a6182cc1daf4f6ca2c2464b6977cde1. This is attempt 1: this message will repeat every 5th retry.
W20260806 07:58:32.460750 6384 consensus_peers.cc:597] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 -> Peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Couldn't send request to peer 0d9e5266c4e04896a714d37cd67fdcc7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbc52165692848699c405d3e54739402. This is attempt 1: this message will repeat every 5th retry.
I20260806 07:58:32.465770 6885 raft_consensus.cc:493] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.466100 6885 raft_consensus.cc:515] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.466336 6544 raft_consensus.cc:493] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.466547 6544 raft_consensus.cc:515] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.466871 6885 leader_election.cc:290] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.467288 6544 leader_election.cc:290] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.467743 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0e900ebca2c44f381f01492b0601036" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.467881 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89cbb223d2446f789aa88d6a6ec778a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.468087 6312 raft_consensus.cc:2468] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:32.468135 7016 ts_tablet_manager.cc:1929] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.468288 7016 log.cc:1199] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/5a6182cc1daf4f6ca2c2464b6977cde1
I20260806 07:58:32.468364 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
W20260806 07:58:32.468575 6113 leader_election.cc:343] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.468706 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.468751 7016 ts_tablet_manager.cc:1950] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.468806 6113 leader_election.cc:304] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters:
I20260806 07:58:32.469050 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.469347 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.469794 7016 tablet_replica.cc:333] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.470009 7016 raft_consensus.cc:2243] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.470218 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet 5a6182cc1daf4f6ca2c2464b6977cde1 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.471159 6544 raft_consensus.cc:2804] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.471252 6544 raft_consensus.cc:493] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.471346 6544 raft_consensus.cc:3060] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.471711 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.472085 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.472378 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.473259 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.473636 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.470446 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89cbb223d2446f789aa88d6a6ec778a" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.474128 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.474117 7016 raft_consensus.cc:2272] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.474462 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.470861 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0e900ebca2c44f381f01492b0601036" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.474596 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
W20260806 07:58:32.475080 6114 leader_election.cc:343] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.475210 6114 leader_election.cc:304] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
W20260806 07:58:32.475637 6114 leader_election.cc:343] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.476078 6885 raft_consensus.cc:2749] T f89cbb223d2446f789aa88d6a6ec778a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:32.477545 6544 raft_consensus.cc:515] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.479278 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0e900ebca2c44f381f01492b0601036" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:32.479998 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.480298 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.480583 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.480968 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.481251 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.481740 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.481815 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0e900ebca2c44f381f01492b0601036" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:32.482158 6312 raft_consensus.cc:3060] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
W20260806 07:58:32.482308 6114 leader_election.cc:343] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.482623 6544 leader_election.cc:290] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.482772 7016 ts_tablet_manager.cc:1916] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.486142 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.017s user 0.005s sys 0.002s Metrics: {"bytes_written":471,"delete_count":0,"lbm_write_time_us":209,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":2}
I20260806 07:58:32.490566 6312 raft_consensus.cc:2468] T f0e900ebca2c44f381f01492b0601036 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:32.490789 7022 tablet_bootstrap.cc:492] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.490957 7022 ts_tablet_manager.cc:1403] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.003s
I20260806 07:58:32.491559 6113 leader_election.cc:304] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 5d4b117032604585af3544cb030a10f2
I20260806 07:58:32.492213 7022 raft_consensus.cc:359] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.492503 7022 raft_consensus.cc:385] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.492612 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.492749 7022 raft_consensus.cc:740] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.492789 6544 raft_consensus.cc:2804] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:32.492946 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.493729 6544 raft_consensus.cc:697] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:32.494153 6544 consensus_queue.cc:237] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.494741 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.021s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":710,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":57,"lbm_read_time_us":187,"lbm_reads_lt_1ms":8,"lbm_write_time_us":165,"lbm_writes_lt_1ms":4,"mutex_wait_us":4111,"reinsert_count":0,"update_count":3}
I20260806 07:58:32.498711 7022 consensus_queue.cc:260] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.499662 7024 tablet_bootstrap.cc:492] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:32.499840 7020 tablet_bootstrap.cc:492] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.499918 7024 ts_tablet_manager.cc:1403] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.171s user 0.020s sys 0.030s
I20260806 07:58:32.500029 7020 ts_tablet_manager.cc:1403] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.088s user 0.005s sys 0.006s
I20260806 07:58:32.500895 7020 raft_consensus.cc:359] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.500942 7024 raft_consensus.cc:359] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.501142 7024 raft_consensus.cc:385] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.501169 7020 raft_consensus.cc:385] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.501204 7024 raft_consensus.cc:740] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.501328 7020 raft_consensus.cc:740] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.501714 7024 consensus_queue.cc:260] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.501781 7020 consensus_queue.cc:260] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.502296 7024 ts_tablet_manager.cc:1434] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 07:58:32.502535 7020 ts_tablet_manager.cc:1434] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 07:58:32.502897 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.022s user 0.003s sys 0.002s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":178,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:32.504171 7017 ts_tablet_manager.cc:1929] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.504179 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.504369 7017 log.cc:1199] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/b13ae96de4eb42088b187c4efc95802a
I20260806 07:58:32.504885 7017 ts_tablet_manager.cc:1950] T b13ae96de4eb42088b187c4efc95802a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.506106 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet b13ae96de4eb42088b187c4efc95802a (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.506318 7022 ts_tablet_manager.cc:1434] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.012s user 0.000s sys 0.003s
I20260806 07:58:32.506677 7022 tablet_bootstrap.cc:492] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.506699 7016 ts_tablet_manager.cc:1929] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260806 07:58:32.506980 7016 log.cc:1199] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/be96d575a672438994b6c6364180ec84
I20260806 07:58:32.507614 7016 ts_tablet_manager.cc:1950] T be96d575a672438994b6c6364180ec84 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.507793 7024 tablet_bootstrap.cc:492] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:32.511646 7016 tablet_replica.cc:333] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.512043 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet be96d575a672438994b6c6364180ec84 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.514665 7016 raft_consensus.cc:2243] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.515206 7016 raft_consensus.cc:2272] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.515496 7022 tablet_bootstrap.cc:654] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.515755 7017 tablet_replica.cc:333] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.515585 6015 catalog_manager.cc:5697] T f0e900ebca2c44f381f01492b0601036 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:32.515983 7017 raft_consensus.cc:2243] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.516341 7016 ts_tablet_manager.cc:1916] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.516383 7017 raft_consensus.cc:2272] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.517040 7017 ts_tablet_manager.cc:1916] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.522356 6544 raft_consensus.cc:493] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.522619 6544 raft_consensus.cc:515] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.522871 7024 tablet_bootstrap.cc:654] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.523942 6544 leader_election.cc:290] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.525664 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f1ada5024c07b124303637baadab" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.526094 6452 raft_consensus.cc:2468] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:32.526830 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.526880 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f1ada5024c07b124303637baadab" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.527338 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.527788 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.528220 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.530841 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.531361 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.531695 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.532044 7016 ts_tablet_manager.cc:1929] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.532330 7016 log.cc:1199] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/dbc52165692848699c405d3e54739402
W20260806 07:58:32.532534 6113 leader_election.cc:343] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.533041 7016 ts_tablet_manager.cc:1950] T dbc52165692848699c405d3e54739402 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.533213 6114 leader_election.cc:304] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:32.534432 6544 raft_consensus.cc:2804] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.534498 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.534210 7016 tablet_replica.cc:333] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.534740 6544 raft_consensus.cc:493] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.534883 7016 raft_consensus.cc:2243] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.534955 6544 raft_consensus.cc:3060] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.535019 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.535293 7016 raft_consensus.cc:2272] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.535496 6002 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.534472 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet dbc52165692848699c405d3e54739402 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.536199 7016 ts_tablet_manager.cc:1916] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.537365 6885 raft_consensus.cc:493] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.537563 6885 raft_consensus.cc:515] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.538688 6885 leader_election.cc:290] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.539402 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e954825b5e44e4ebd01e742b8ff0bf9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.539997 7020 tablet_bootstrap.cc:492] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.539496 6544 raft_consensus.cc:515] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
W20260806 07:58:32.541191 6113 leader_election.cc:343] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.542079 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e954825b5e44e4ebd01e742b8ff0bf9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.542302 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f1ada5024c07b124303637baadab" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:32.542490 6544 leader_election.cc:290] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
W20260806 07:58:32.543247 6113 leader_election.cc:343] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.543921 6452 raft_consensus.cc:2468] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 0.
I20260806 07:58:32.542192 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f1ada5024c07b124303637baadab" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:32.544654 6453 raft_consensus.cc:3060] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.545614 6918 raft_consensus.cc:493] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.545814 6918 raft_consensus.cc:515] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.546754 6918 leader_election.cc:290] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.547363 6114 leader_election.cc:304] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:32.547595 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6828e964317544099e1886f4398a0322" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:32.547886 6178 raft_consensus.cc:2393] T 6828e964317544099e1886f4398a0322 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d9e5266c4e04896a714d37cd67fdcc7 in current term 1: Already voted for candidate 5d4b117032604585af3544cb030a10f2 in this term.
I20260806 07:58:32.548619 7022 tablet_bootstrap.cc:492] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.548832 7022 ts_tablet_manager.cc:1403] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.000s
I20260806 07:58:32.550084 7022 raft_consensus.cc:359] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.550356 7022 raft_consensus.cc:385] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.550462 7022 raft_consensus.cc:740] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.550808 7022 consensus_queue.cc:260] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.551237 6885 raft_consensus.cc:2804] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.551366 7022 ts_tablet_manager.cc:1434] T 61f1edf7136a409bb7e4232596b305b0 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 07:58:32.551364 6885 raft_consensus.cc:493] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.551638 6885 raft_consensus.cc:3060] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.551710 7022 tablet_bootstrap.cc:492] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.556833 7022 tablet_bootstrap.cc:654] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.556911 6623 raft_consensus.cc:493] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.557236 6885 raft_consensus.cc:515] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.557432 6623 raft_consensus.cc:515] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.558359 6623 leader_election.cc:290] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:32.558889 6312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e954825b5e44e4ebd01e742b8ff0bf9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7"
I20260806 07:58:32.559242 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7262a4ea974a96be2234607b60b179" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:32.559695 6113 leader_election.cc:343] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.560542 6885 leader_election.cc:290] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Requested vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.560760 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c7262a4ea974a96be2234607b60b179" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
W20260806 07:58:32.561782 6384 leader_election.cc:343] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.556619 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.028s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":229,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
W20260806 07:58:32.565439 6387 leader_election.cc:343] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.565734 6387 leader_election.cc:304] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d4b117032604585af3544cb030a10f2; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:32.565193 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6828e964317544099e1886f4398a0322" candidate_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.567634 6623 raft_consensus.cc:2749] T 2c7262a4ea974a96be2234607b60b179 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:32.568388 6250 leader_election.cc:304] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d9e5266c4e04896a714d37cd67fdcc7; no voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168
I20260806 07:58:32.568900 6918 raft_consensus.cc:3060] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.563478 7017 ts_tablet_manager.cc:1929] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.570182 7017 log.cc:1199] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/5a6182cc1daf4f6ca2c2464b6977cde1
I20260806 07:58:32.570899 7017 ts_tablet_manager.cc:1950] T 5a6182cc1daf4f6ca2c2464b6977cde1 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.571720 6005 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.036s user 0.009s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":738,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":58,"lbm_read_time_us":184,"lbm_reads_lt_1ms":8,"lbm_write_time_us":179,"lbm_writes_lt_1ms":4,"mutex_wait_us":14128,"reinsert_count":0,"update_count":4}
I20260806 07:58:32.572271 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet 5a6182cc1daf4f6ca2c2464b6977cde1 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.573050 6918 raft_consensus.cc:2749] T 6828e964317544099e1886f4398a0322 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:32.574412 6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e954825b5e44e4ebd01e742b8ff0bf9" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2"
I20260806 07:58:32.574702 6452 raft_consensus.cc:3060] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.574937 7017 tablet_replica.cc:333] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.575253 7017 raft_consensus.cc:2243] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.575557 7017 raft_consensus.cc:2272] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.575769 6885 raft_consensus.cc:493] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.576093 6885 raft_consensus.cc:515] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.576222 7020 tablet_bootstrap.cc:654] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.577396 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a3d34e6ca143c7834aaa4a9a17b2c3" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
W20260806 07:58:32.578101 6113 leader_election.cc:343] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.578356 6453 raft_consensus.cc:2468] T e734f1ada5024c07b124303637baadab P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:32.578421 7016 ts_tablet_manager.cc:1929] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.578569 6885 leader_election.cc:290] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979), 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665)
I20260806 07:58:32.578832 7016 log.cc:1199] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/c31bd22c7f6e47c595747dec77a8cc70
I20260806 07:58:32.579555 6114 leader_election.cc:304] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:32.579972 6885 raft_consensus.cc:2804] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:32.580224 7016 ts_tablet_manager.cc:1950] T c31bd22c7f6e47c595747dec77a8cc70 P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.580756 6885 raft_consensus.cc:697] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:32.581175 6885 consensus_queue.cc:237] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.581804 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet c31bd22c7f6e47c595747dec77a8cc70 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.582258 7016 tablet_replica.cc:333] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: stopping tablet replica
I20260806 07:58:32.582682 6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a3d34e6ca143c7834aaa4a9a17b2c3" candidate_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4b117032604585af3544cb030a10f2" is_pre_election: true
I20260806 07:58:32.582839 7016 raft_consensus.cc:2243] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 07:58:32.583333 7016 raft_consensus.cc:2272] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 07:58:32.583412 6114 leader_election.cc:343] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.583590 6114 leader_election.cc:304] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7, 5d4b117032604585af3544cb030a10f2
I20260806 07:58:32.583889 6954 raft_consensus.cc:2749] T c0a3d34e6ca143c7834aaa4a9a17b2c3 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 07:58:32.584252 6452 raft_consensus.cc:2468] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 5d4b117032604585af3544cb030a10f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7793159fc1ab4bd1b2a50dbd3b3e5168 in term 1.
I20260806 07:58:32.584554 7016 ts_tablet_manager.cc:1916] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.584820 6114 leader_election.cc:304] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:32.585171 6954 raft_consensus.cc:2804] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 07:58:32.585326 6954 raft_consensus.cc:697] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 LEADER]: Becoming Leader. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Running, Role: LEADER
I20260806 07:58:32.585572 6954 consensus_queue.cc:237] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.598814 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.117s user 0.009s sys 0.021s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":680,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":70218,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":1396,"lbm_read_time_us":225,"lbm_reads_lt_1ms":8,"lbm_write_time_us":223,"lbm_writes_lt_1ms":4,"mutex_wait_us":21386,"reinsert_count":0,"thread_start_us":69961,"threads_started":1,"update_count":2}
I20260806 07:58:32.599643 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.603120 7022 tablet_bootstrap.cc:492] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.603437 7022 ts_tablet_manager.cc:1403] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.052s user 0.013s sys 0.013s
I20260806 07:58:32.604239 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.111s user 0.004s sys 0.003s Metrics: {"bytes_written":471,"delete_count":0,"lbm_write_time_us":274,"lbm_writes_lt_1ms":4,"mutex_wait_us":103396,"reinsert_count":0,"update_count":2}
I20260806 07:58:32.604676 5929 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.100s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":99942}
I20260806 07:58:32.604743 7016 ts_tablet_manager.cc:1929] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.605013 7016 log.cc:1199] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-2/wal/wals/f9aa031a25104549905f7a7e1007187f
I20260806 07:58:32.604394 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.111s user 0.000s sys 0.000s Metrics: {"mutex_wait_us":110828}
I20260806 07:58:32.606287 7024 tablet_bootstrap.cc:492] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:32.606539 7024 ts_tablet_manager.cc:1403] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.099s user 0.004s sys 0.010s
I20260806 07:58:32.607650 7024 raft_consensus.cc:359] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.607870 7024 raft_consensus.cc:385] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.607959 7024 raft_consensus.cc:740] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.610399 7022 raft_consensus.cc:359] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.612193 7016 ts_tablet_manager.cc:1950] T f9aa031a25104549905f7a7e1007187f P 5d4b117032604585af3544cb030a10f2: Deleting consensus metadata
I20260806 07:58:32.613075 7020 tablet_bootstrap.cc:492] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2: No bootstrap required, opened a new log
I20260806 07:58:32.613353 7020 ts_tablet_manager.cc:1403] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.011s
I20260806 07:58:32.613787 7024 consensus_queue.cc:260] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.614104 7022 raft_consensus.cc:385] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.614646 7020 raft_consensus.cc:359] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.614915 7020 raft_consensus.cc:385] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.614970 7022 raft_consensus.cc:740] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.615125 7020 raft_consensus.cc:740] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4b117032604585af3544cb030a10f2, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.615449 7020 consensus_queue.cc:260] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.615413 7022 consensus_queue.cc:260] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.616287 7020 ts_tablet_manager.cc:1434] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 07:58:32.616650 7020 tablet_bootstrap.cc:492] T 61f1edf7136a409bb7e4232596b305b0 P 5d4b117032604585af3544cb030a10f2: Bootstrap starting.
I20260806 07:58:32.617110 7024 ts_tablet_manager.cc:1434] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260806 07:58:32.617440 7022 ts_tablet_manager.cc:1434] T 50c57f5c236c49f1a2cffa660a9854da P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.014s user 0.002s sys 0.001s
I20260806 07:58:32.617462 7024 tablet_bootstrap.cc:492] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:32.617998 7022 tablet_bootstrap.cc:492] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.619047 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.019s user 0.008s sys 0.000s Metrics: {"cfile_cache_miss":4,"cfile_cache_miss_bytes":741,"cfile_init":1,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":244,"lbm_reads_lt_1ms":8,"lbm_write_time_us":161,"lbm_writes_lt_1ms":4,"mutex_wait_us":5616,"reinsert_count":0,"update_count":4}
I20260806 07:58:32.619799 7024 tablet_bootstrap.cc:654] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.613080 7017 ts_tablet_manager.cc:1916] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.622998 6017 catalog_manager.cc:5697] T e734f1ada5024c07b124303637baadab P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:32.625306 7022 tablet_bootstrap.cc:654] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.632997 5999 catalog_manager.cc:5028] TS 5d4b117032604585af3544cb030a10f2 (127.20.202.195:36665): tablet f9aa031a25104549905f7a7e1007187f (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.635322 7024 tablet_bootstrap.cc:492] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7: No bootstrap required, opened a new log
I20260806 07:58:32.635493 7024 ts_tablet_manager.cc:1403] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260806 07:58:32.636246 7024 raft_consensus.cc:359] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.636416 7024 raft_consensus.cc:385] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.636466 7024 raft_consensus.cc:740] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9e5266c4e04896a714d37cd67fdcc7, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.636711 7024 consensus_queue.cc:260] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.638175 7022 tablet_bootstrap.cc:492] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.638254 7024 ts_tablet_manager.cc:1434] T 50c57f5c236c49f1a2cffa660a9854da P 0d9e5266c4e04896a714d37cd67fdcc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:32.638377 7017 ts_tablet_manager.cc:1929] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260806 07:58:32.638514 7017 log.cc:1199] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/be96d575a672438994b6c6364180ec84
I20260806 07:58:32.638546 7024 tablet_bootstrap.cc:492] T f4d84432a5b54420ab064ff63612b8bc P 0d9e5266c4e04896a714d37cd67fdcc7: Bootstrap starting.
I20260806 07:58:32.638979 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.639061 7022 ts_tablet_manager.cc:1403] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.000s
I20260806 07:58:32.639489 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.640075 7022 raft_consensus.cc:359] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.640300 7022 raft_consensus.cc:385] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.640239 6004 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.640380 7022 raft_consensus.cc:740] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.640681 7022 consensus_queue.cc:260] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.640785 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.640944 7024 tablet_bootstrap.cc:654] T f4d84432a5b54420ab064ff63612b8bc P 0d9e5266c4e04896a714d37cd67fdcc7: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.641321 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.641672 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.642166 7022 ts_tablet_manager.cc:1434] T 0c515ba9095b4a13bc21eea1bed0ae7a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 07:58:32.642367 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.642576 7022 tablet_bootstrap.cc:492] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.642627 6003 maintenance_manager.cc:643] P 15b584d20bab4416a9d975df6675b207: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.645203 7022 tablet_bootstrap.cc:654] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.646528 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.646900 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.647358 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.647631 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.648310 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.648667 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.648882 5930 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.649232 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.649680 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260806 07:58:32.650095 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260806 07:58:32.650668 7017 ts_tablet_manager.cc:1950] T be96d575a672438994b6c6364180ec84 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.653584 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet be96d575a672438994b6c6364180ec84 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.654078 7017 tablet_replica.cc:333] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.654373 7017 raft_consensus.cc:2243] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.654670 7017 raft_consensus.cc:2272] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.655454 7017 ts_tablet_manager.cc:1916] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.657114 6017 catalog_manager.cc:5697] T 4e954825b5e44e4ebd01e742b8ff0bf9 P 7793159fc1ab4bd1b2a50dbd3b3e5168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193). New cstate: current_term: 1 leader_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } health_report { overall_health: UNKNOWN } } }
I20260806 07:58:32.666157 7017 ts_tablet_manager.cc:1929] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.666472 7017 log.cc:1199] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/dbc52165692848699c405d3e54739402
I20260806 07:58:32.667147 7017 ts_tablet_manager.cc:1950] T dbc52165692848699c405d3e54739402 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.668530 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet dbc52165692848699c405d3e54739402 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
I20260806 07:58:32.670192 7022 tablet_bootstrap.cc:492] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.670442 7022 ts_tablet_manager.cc:1403] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.001s
I20260806 07:58:32.672109 7022 raft_consensus.cc:359] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.672544 7022 raft_consensus.cc:385] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.672847 7022 raft_consensus.cc:740] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.673326 7022 consensus_queue.cc:260] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.674741 7022 ts_tablet_manager.cc:1434] T f4d84432a5b54420ab064ff63612b8bc P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 07:58:32.675148 7022 tablet_bootstrap.cc:492] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Bootstrap starting.
I20260806 07:58:32.678797 7017 tablet_replica.cc:333] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: stopping tablet replica
I20260806 07:58:32.679608 7017 raft_consensus.cc:2243] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 07:58:32.679922 7017 raft_consensus.cc:2272] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 07:58:32.680850 7017 ts_tablet_manager.cc:1916] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 07:58:32.681414 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.679378 7022 tablet_bootstrap.cc:654] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.681895 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.684623 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.685189 6006 maintenance_manager.cc:419] P 15b584d20bab4416a9d975df6675b207: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.692418 7017 ts_tablet_manager.cc:1929] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 07:58:32.692689 7017 log.cc:1199] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting WAL directory at /tmp/dist-test-taskUU61cc/test-tmp/external_mini_cluster-test.0.Kudu3762Test.AddMaster.1786002471851738-21291-0/minicluster-data/ts-0/wal/wals/c31bd22c7f6e47c595747dec77a8cc70
I20260806 07:58:32.693281 7017 ts_tablet_manager.cc:1950] T c31bd22c7f6e47c595747dec77a8cc70 P 7793159fc1ab4bd1b2a50dbd3b3e5168: Deleting consensus metadata
I20260806 07:58:32.695442 5931 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.048s user 0.006s sys 0.000s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":195,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":1}
I20260806 07:58:32.695734 7022 tablet_bootstrap.cc:492] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168: No bootstrap required, opened a new log
I20260806 07:58:32.695988 7022 ts_tablet_manager.cc:1403] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.003s
I20260806 07:58:32.696354 5932 maintenance_manager.cc:419] P a3d2d029e02f42bdb166bf54079a7128: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20260806 07:58:32.697227 7022 raft_consensus.cc:359] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.697510 7022 raft_consensus.cc:385] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 07:58:32.697623 7022 raft_consensus.cc:740] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793159fc1ab4bd1b2a50dbd3b3e5168, State: Initialized, Role: FOLLOWER
I20260806 07:58:32.698119 6623 raft_consensus.cc:493] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 07:58:32.698071 7022 consensus_queue.cc:260] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.698529 6623 raft_consensus.cc:515] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" member_type: VOTER last_known_addr { host: "127.20.202.193" port: 45079 } } peers { permanent_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" member_type: VOTER last_known_addr { host: "127.20.202.194" port: 32979 } } peers { permanent_uuid: "5d4b117032604585af3544cb030a10f2" member_type: VOTER last_known_addr { host: "127.20.202.195" port: 36665 } }
I20260806 07:58:32.699693 6623 leader_election.cc:290] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079), 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979)
I20260806 07:58:32.700294 7020 tablet_bootstrap.cc:654] T 61f1edf7136a409bb7e4232596b305b0 P 5d4b117032604585af3544cb030a10f2: Neither blocks nor log segments found. Creating new log.
I20260806 07:58:32.700472 6315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f789d0955264a6faefbd09df6964b5a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9e5266c4e04896a714d37cd67fdcc7" is_pre_election: true
I20260806 07:58:32.701071 6178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f789d0955264a6faefbd09df6964b5a" candidate_uuid: "5d4b117032604585af3544cb030a10f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7793159fc1ab4bd1b2a50dbd3b3e5168" is_pre_election: true
I20260806 07:58:32.701272 6178 raft_consensus.cc:2468] T 1f789d0955264a6faefbd09df6964b5a P 7793159fc1ab4bd1b2a50dbd3b3e5168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4b117032604585af3544cb030a10f2 in term 0.
I20260806 07:58:32.702474 6001 catalog_manager.cc:5028] TS 7793159fc1ab4bd1b2a50dbd3b3e5168 (127.20.202.193:45079): tablet c31bd22c7f6e47c595747dec77a8cc70 (table table-3 [id=8ee7655d8a2b45aab10febb2f9465dfb]) successfully deleted
W20260806 07:58:32.704861 6384 leader_election.cc:343] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d9e5266c4e04896a714d37cd67fdcc7 (127.20.202.194:32979): Illegal state: must be running to vote when last-logged opid is not known
I20260806 07:58:32.705681 6387 leader_election.cc:304] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d4b117032604585af3544cb030a10f2, 7793159fc1ab4bd1b2a50dbd3b3e5168; no voters: 0d9e5266c4e04896a714d37cd67fdcc7
I20260806 07:58:32.706372 6623 raft_consensus.cc:2804] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 07:58:32.706506 6623 raft_consensus.cc:493] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 07:58:32.706674 6623 raft_consensus.cc:3060] T 1f789d0955264a6faefbd09df6964b5a P 5d4b117032604585af3544cb030a10f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 07:58:32.706809 5928 maintenance_manager.cc:643] P a3d2d029e02f42bdb166bf54079a7128: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.002s sys 0.002s Metrics: {"bytes_written":441,"delete_count":0,"lbm_write_time_us":181,